00010000:00010000:3.0:1788572537.657499:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572537.657503:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53800, rc 0) 00000100:00100000:3.0:1788572537.657556:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+157:35867:x1875453360201600:12345-192.168.202.40@tcp:101:diff.0 Request processed in 424us (757us total) trans 4294970035 rc 0/0 00000100:00100000:3.0:1788572537.657563:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3612 00000100:00100000:3.0:1788572537.657584:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3613 00000100:00100000:3.0:1788572537.657591:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35878:x1875453360201856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572537.657600:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572537.657608:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160bdd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eef0a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657652:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657663:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657670:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572537.657677:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x77:0x0]: returning size 3 00010000:00010000:3.0:1788572537.657715:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657723:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657728:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657733:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.657738:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657746:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657756:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572537.657758:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572537.657762:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657767:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160be4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657792:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160bdd lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef0a expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657801:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572537.657803:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572537.657807:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160bdd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef0a expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.657815:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:3.0:1788572537.657836:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35878:x1875453360201856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 247us (913us total) trans 4294970036 rc 301/301 00000100:00100000:3.0:1788572537.657843:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3613 00000100:00100000:3.0:1788572537.663583:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360202112 00000100:00100000:3.0:1788572537.663592:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360202112 00000100:00100000:3.0:1788572537.663648:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3615 00000100:00100000:3.0:1788572537.663657:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35879:x1875453360202112:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572537.663691:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x7b:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572537.663792:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35879:x1875453360202112:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 136us (295us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.663800:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3615 00000100:00100000:3.0:1788572537.664655:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360202368 00000100:00100000:3.0:1788572537.664661:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360202368 00002000:00100000:3.0:1788572537.664697:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x3a:0x0] (0->4095) req@ffff9626c4920380 x1875453360202368/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:198/0 lens 488/0 e 0 to 0 dl 1788572548 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.664708:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4920380 with x1875453360202368 ext(0->4095) 00002000:00010000:3.0:1788572537.664714:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4920380 00000100:00100000:3.0:1788572537.664738:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 207 00000100:00100000:3.0:1788572537.664746:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+28:2348:x1875453360202368:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572537.664953:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+28:2348:x1875453360202368:12345-192.168.202.40@tcp:3:diff.0 Request processed in 199us (382us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572537.664961:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 207 00002000:00100000:3.0:1788572537.664967:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x3a:0x0] (0->4095) req@ffff9626c4920380 x1875453360202368/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:198/0 lens 488/4536 e 0 to 0 dl 1788572548 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.664976:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4920380 with x1875453360202368 ext(0->4095) 00002000:00010000:3.0:1788572537.664979:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4920380 00000100:00100000:3.0:1788572537.667787:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360202496 00000100:00100000:3.0:1788572537.667798:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360202496 00000100:00100000:3.0:1788572537.667863:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 292 00000100:00100000:3.0:1788572537.667874:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+157:2350:x1875453360202496:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572537.667891:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572537.667916:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160bcf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x68:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef03 expref: 157 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.667937:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x68:0x0].0x0 (ffff9627c56ff900) 00010000:00010000:3.0:1788572537.667951:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572537.667996:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+156:2350:x1875453360202496:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 124us (448us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.668028:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 292 00000100:00100000:3.0:1788572537.669524:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382323968 00000100:00100000:3.0:1788572537.669530:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382323968 00000100:00100000:3.0:1788572537.669580:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1788572537.669588:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1875453382323968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572537.679347:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1875453382323968:12345-0@lo:400:kworker.0 Request processed in 9636us (10141us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.679361:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1788572537.705502:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360203008 00000100:00100000:3.0:1788572537.705512:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360203008 00000100:00100000:3.0:1788572537.705608:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 821 00000100:00100000:3.0:1788572537.705620:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+156:35867:x1875453360203008:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572537.706040:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+157:35867:x1875453360203008:12345-192.168.202.40@tcp:35:diff.0 Request processed in 421us (649us total) trans 4294970038 rc 0/0 00000100:00100000:3.0:1788572537.706049:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 821 00000100:00100000:3.0:1788572537.713741:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360203776 00000100:00100000:3.0:1788572537.713752:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360203776 00000100:00100000:3.0:1788572537.713823:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360203904 00000100:00100000:3.0:1788572537.713826:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360203904 00000100:00100000:3.0:1788572537.718432:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360204160 00000100:00100000:3.0:1788572537.718440:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360204160 00000100:00100000:3.0:1788572537.718496:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3624 00000100:00100000:3.0:1788572537.718506:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec193850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35878:x1875453360204160:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572537.718526:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572537.718540:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160ca8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eef49 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718592:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718604:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718615:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718628:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718635:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718642:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718923:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb160cbd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718933:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb160cbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718941:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb160cbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572537.718962:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:3.0:1788572537.718965:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572537.718979:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb160cbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718988:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.718995:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719024:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719031:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.719035:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719041:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719052:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572537.719054:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572537.719058:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719064:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160caf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719075:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719085:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719090:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719095:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.719098:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719104:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719111:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627c5716900) 00010000:00010000:3.0:1788572537.719113:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572537.719116:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719121:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160cb6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719145:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160ca8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef49 expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719157:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb160cbd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef49 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.719164:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572537.719166:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572537.719169:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:3.0:1788572537.721162:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec193850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35878:x1875453360204160:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2656us (2804us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.721171:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3624 00000100:00100000:3.0:1788572537.748464:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.749099:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.749346:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572537.752833:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb160d34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752842:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb160d34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572537.752881:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:3.0:1788572537.752885:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572537.752902:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb160d34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752913:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752920:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752926:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752931:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.752935:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752941:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752951:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572537.752954:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572537.752957:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752963:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb160d26 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752973:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752979:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752985:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.752991:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.752994:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.753000:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.753034:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627c2abc400) 00010000:00010000:3.0:1788572537.753036:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572537.753039:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.753045:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160d2d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.753081:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac600/0x85ace67feb160d1f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef65 expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.753094:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb160d34 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef65 expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.753102:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572537.753104:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572537.753106:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679400, rc 0) 00000100:00100000:3.0:1788572537.753140:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35879:x1875453360205440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9246us (10380us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.753147:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3629 00010000:00010000:3.0:1788572537.756621:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.756632:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.756639:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.756644:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bace00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.756649:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.756657:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.756672:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x69:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:3.0:1788572537.756675:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1788572537.756680:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.756686:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb160d18 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x69:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572537.758414:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.758595:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.758631:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.758655:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360206336 00000100:00100000:3.0:1788572537.758662:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360206336 00000100:00100000:3.0:1788572537.758705:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 293 00000100:00100000:3.0:1788572537.758715:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:2349:x1875453360206336:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572537.758728:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572537.758747:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb160d0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x67:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef5e expref: 159 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.758763:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x67:0x0].0x0 (ffff9627d0ee2d00) 00010000:00010000:3.0:1788572537.758767:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572537.758802:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+158:2349:x1875453360206336:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 88us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.758809:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 293 00000100:00100000:3.0:1788572537.758859:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.771561:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360206976 00000100:00100000:3.0:1788572537.771578:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360206976 00000100:00100000:3.0:1788572537.771693:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 887 00000100:00100000:3.0:1788572537.771704:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360206976:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572537.772222:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360206976:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 517us (2278us total) trans 4294967515 rc 0/0 00000100:00100000:3.0:1788572537.772241:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 887 00000004:00010000:3.0:1788572537.781245:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000080c4199e reply state 00000000c851641c transno 4294970045 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb160d81 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572537.781319:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572537.781444:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35879:x1875453360207744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2784us (3616us total) trans 4294970045 rc 0/0 00000100:00100000:3.0:1788572537.781452:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3638 00000100:00100000:3.0:1788572537.783692:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360208256 00000100:00100000:3.0:1788572537.783702:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360208256 00000100:00100000:3.0:1788572537.783760:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 888 00000100:00100000:3.0:1788572537.783770:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360208256:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572537.784079:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360208256:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 310us (1190us total) trans 4294967517 rc 0/0 00000100:00100000:3.0:1788572537.784087:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 888 00000100:00100000:3.0:1788572537.784116:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360207872 00000100:00100000:3.0:1788572537.784120:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360207872 00000100:00100000:3.0:1788572537.784154:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3639 00000100:00100000:3.0:1788572537.784160:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:35867:x1875453360207872:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572537.784177:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572537.784196:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160d88 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eef88 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784614:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784630:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784642:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784658:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784665:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784672:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784921:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160d9d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x66:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784930:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160d9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x66:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.784938:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160d9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x66:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572537.785142:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x66:0x0]: returning size 3 00000004:00200000:3.0:1788572537.785147:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572537.785166:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160d9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x66:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785178:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160d9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x66:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785190:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785197:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785203:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785209:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.785213:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785221:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785339:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0x0 (ffff9627d20cbd00) 00010000:00010000:3.0:1788572537.785342:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572537.785348:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785354:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160d8f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785364:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785370:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785375:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785380:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572537.785382:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785388:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785394:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0xbd5240a3 (ffff9627c5716900) 00010000:00010000:3.0:1788572537.785396:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572537.785398:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785403:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160d96 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x60:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785434:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160d88 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef88 expref: 161 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785445:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53800/0x85ace67feb160d9d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x66:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef88 expref: 160 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.785453:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x66:0x0].0x0 (ffff9627c5716800) 00010000:00010000:3.0:1788572537.785455:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572537.785457:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53800, rc 0) 00000100:00100000:3.0:1788572537.785574:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+160:35867:x1875453360207872:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1413us (2760us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.785582:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3639 00000100:00100000:3.0:1788572537.788519:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.800884:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.800961:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.800995:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.802461:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.809641:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.809676:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.809704:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.811414:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3645 00000100:00100000:3.0:1788572537.811425:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35878:x1875453360209664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572537.812703:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572537.812725:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160dc7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eefa4 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.812790:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160dce lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.812806:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160dce lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.812818:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160dce lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572537.812851:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572537.812868:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160dce lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.812889:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb160dc7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefa4 expref: 101 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.812902:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160dce lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefa4 expref: 100 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.812911:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6a:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:3.0:1788572537.812914:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572537.812918:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:3.0:1788572537.813068:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35878:x1875453360209664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1644us (3389us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.813076:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3645 00000100:00100000:3.0:1788572537.824241:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.824324:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.824354:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.824390:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.824421:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360210304 00000100:00100000:3.0:1788572537.824429:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360210304 00000100:00100000:3.0:1788572537.825166:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360210816 00000100:00100000:3.0:1788572537.825172:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360210816 00000100:00100000:3.0:1788572537.825196:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 891 00000100:00100000:3.0:1788572537.825204:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360210304:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572537.825231:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572537.825260:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb160de3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eefb2 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572537.825292:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7379, blocks: 0x1 00010000:00010000:3.0:1788572537.825311:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3c:0x0].0x0 since=885s v=(0 0) 00010000:00010000:3.0:1788572537.825316:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572537.825326:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb160de3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefb2 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.825354:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb160de3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefb2 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.825369:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:3.0:1788572537.825396:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360210304:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 193us (1071us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.825402:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 891 00000100:00100000:3.0:1788572537.825418:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 892 00000100:00100000:3.0:1788572537.825424:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360210816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572537.825433:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572537.825445:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000702c2999/0x85ace67feb160dea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eefb9 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572537.825462:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x3c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7379, blocks: 0x1 00010000:00010000:3.0:1788572537.825679:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3c:0x0].0x0 since=885s v=(0 0) 00010000:00010000:3.0:1788572537.825684:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572537.825690:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000702c2999/0x85ace67feb160dea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefb9 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.825704:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000702c2999/0x85ace67feb160dea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefb9 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572537.825714:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:3.0:1788572537.825739:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360210816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 315us (1349us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.825745:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 892 00000100:00100000:3.0:1788572537.834173:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.835432:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 209 00000100:00100000:3.0:1788572537.835446:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+40:2350:x1875453360210944:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572537.835725:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+40:2350:x1875453360210944:12345-192.168.202.40@tcp:3:diff.0 Request processed in 282us (1563us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572537.835733:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 209 00002000:00100000:3.0:1788572537.835740:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x37:0x0] (0->4095) req@ffff9626c10a3480 x1875453360210944/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:198/0 lens 488/4536 e 0 to 0 dl 1788572548 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.835749:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a3480 with x1875453360210944 ext(0->4095) 00002000:00010000:3.0:1788572537.835753:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c10a3480 00000100:00100000:3.0:1788572537.861221:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360212864 00000100:00100000:3.0:1788572537.861225:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360212864 00002000:00100000:3.0:1788572537.861255:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x3c:0x0] (0->2) req@ffff9627e786f100 x1875453360212864/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:198/0 lens 496/0 e 0 to 0 dl 1788572548 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.861265:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786f100 with x1875453360212864 ext(0->2) 00002000:00010000:3.0:1788572537.861270:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786f100 00000100:00100000:3.0:1788572537.861286:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 210 00000100:00100000:3.0:1788572537.861292:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360212864:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572537.861663:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2347:x1875453360212864:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 370us (864us total) trans 4294967517 rc 0/0 00000100:00100000:3.0:1788572537.861671:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 210 00002000:00100000:3.0:1788572537.861679:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x3c:0x0] (0->2) req@ffff9627e786f100 x1875453360212864/t4294967517(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:198/0 lens 496/448 e 0 to 0 dl 1788572548 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.861686:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786f100 with x1875453360212864 ext(0->2) 00002000:00010000:3.0:1788572537.861690:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786f100 00000100:00100000:3.0:1788572537.861736:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382324480:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572537.861754:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572537.861772:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360212224:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6566us (6796us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.861780:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 864 00000100:00100000:3.0:1788572537.863675:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.866780:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360213120 00000100:00100000:3.0:1788572537.866789:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360213120 00000100:00100000:3.0:1788572537.866843:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3652 00000100:00100000:3.0:1788572537.867114:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35877:x1875453360213120:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572537.873075:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382324736:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572537.873096:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572537.873155:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360212992:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 9235us (9486us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.873165:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 865 00000100:00100000:3.0:1788572537.895102:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382324992:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572537.895127:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572537.895186:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360213888:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7585us (8387us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.895198:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 866 00000100:00100000:3.0:1788572537.895280:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360214656 00000100:00100000:3.0:1788572537.895285:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360214656 00002000:00100000:3.0:1788572537.895312:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x3d:0x0] (0->2) req@ffff9627d131c380 x1875453360214656/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:198/0 lens 496/0 e 0 to 0 dl 1788572548 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.895321:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131c380 with x1875453360214656 ext(0->2) 00002000:00010000:3.0:1788572537.895326:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131c380 00000100:00100000:3.0:1788572537.895344:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 212 00000100:00100000:3.0:1788572537.895351:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360214656:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572537.895942:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360214656:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 590us (1257us total) trans 4294967518 rc 0/0 00000100:00100000:3.0:1788572537.895950:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 212 00002000:00100000:3.0:1788572537.895957:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x3d:0x0] (0->2) req@ffff9627d131c380 x1875453360214656/t4294967518(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:198/0 lens 496/448 e 0 to 0 dl 1788572548 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572537.895966:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131c380 with x1875453360214656 ext(0->2) 00002000:00010000:3.0:1788572537.895969:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131c380 00000100:00100000:3.0:1788572537.901349:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.906098:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.910041:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.910156:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572537.977376:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35877:x1875453360213120:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 110259us (111797us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572537.977389:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3652 00002000:00010000:3.0:1788572538.004358:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572538.004362:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb mtime from disk: 0 -> 1788572522 00002000:00010000:3.0:1788572538.004364:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb atime from disk: 0 -> 1788572522 00002000:00010000:3.0:1788572538.004366:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb ctime from disk: 1788572537 -> 1788572523 00002000:00010000:3.0:1788572538.004370:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572538.004438:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360217088:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 444us (700us total) trans 4294967519 rc 0/0 00000100:00100000:3.0:1788572538.004446:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 904 00000100:00100000:3.0:1788572538.013402:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360217472 00000100:00100000:3.0:1788572538.013413:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360217472 00010000:00010000:3.0:1788572538.013456:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb160d49 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef6c expref: 67 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.013504:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 867 00000100:00100000:3.0:1788572538.013515:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360217472:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.013531:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.013538:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb160d49 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef6c expref: 67 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.013552:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb160d49 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef6c expref: 67 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.013574:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb160d49 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef6c expref: 67 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.013591:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6a:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:3.0:1788572538.013597:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb160f25 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.013607:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb160f25 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.013618:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb160f25 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.013650:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.013698:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360217472:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 185us (518us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.013705:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 867 00000100:00100000:3.0:1788572538.021874:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360217984 00000100:00100000:3.0:1788572538.021879:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360217984 00010000:00010000:3.0:1788572538.021903:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb160cf5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x69:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef57 expref: 65 pid: 9040 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.022302:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 868 00000100:00100000:3.0:1788572538.022311:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360217984:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.022334:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.022342:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb160cf5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x69:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef57 expref: 65 pid: 9040 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.022355:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb160cf5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x69:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef57 expref: 65 pid: 9040 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.022375:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb160cf5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x69:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef57 expref: 65 pid: 9040 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.022390:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x69:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:3.0:1788572538.022395:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac000/0x85ace67feb160f33 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x69:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.022404:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac000/0x85ace67feb160f33 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.022412:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac000/0x85ace67feb160f33 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.022431:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.022466:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360217984:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 157us (1922us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.022483:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 868 00010000:00010000:3.0:1788572538.033929:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb160d73 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef7a expref: 63 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.033944:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb160d73 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef7a expref: 63 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.033959:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382325504:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.034066:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff5483c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.034452:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572538.035961:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb160e0d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefd5 expref: 12 pid: 6682 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.036039:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 869 00000100:00100000:3.0:1788572538.036050:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360218496:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.036067:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.036092:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb160e0d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefd5 expref: 12 pid: 6682 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.036106:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb160e0d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefd5 expref: 12 pid: 6682 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.036127:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb160e0d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eefd5 expref: 12 pid: 6682 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.036144:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572538.036148:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572538.036154:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002193a230/0x85ace67feb160f3a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.036164:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002193a230/0x85ace67feb160f3a lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.036195:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.036240:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360218496:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 193us (1788us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.036246:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 869 00000100:00100000:3.0:1788572538.047792:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382325504:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.047841:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb160f41 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.048430:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.054502:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.059599:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.062082:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.062125:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.066611:0:5838:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967516 gen 2 00000004:00080000:3.0:1788572538.066618:0:5838:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c13bce00, transno 4294967510 00000004:00080000:3.0:1788572538.066642:0:5838:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c13bdc00, transno 4294967511 00000004:00080000:3.0:1788572538.066645:0:5838:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786ca80, transno 4294967512 00000004:00080000:3.0:1788572538.066649:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967515 committed transno 4294967516 00000004:00080000:3.0:1788572538.066667:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967516 committed transno 4294967516 00000100:00100000:3.0:1788572538.066681:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5432d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382326016:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1788572538.067714:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970060 00000100:00100000:3.0:1788572538.067781:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382326144:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572538.067824:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572538.068670:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb59c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.072754:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382326144:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1788572538.072819:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072830:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072838:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072843:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.072847:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072857:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072879:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6a:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:3.0:1788572538.072881:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572538.072888:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072893:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb160f56 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072931:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072938:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072956:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072961:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.072964:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072971:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072980:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572538.072982:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572538.072985:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.072990:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb15e954 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073181:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073194:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073204:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073212:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073218:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073234:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073238:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.073241:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073246:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073255:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572538.073257:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.073260:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073265:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160f64 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x77:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073276:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073282:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073288:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073293:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.073296:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073301:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073308:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572538.073310:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572538.073312:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.073318:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb160f4f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.073572:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35878:x1875453360219008:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 18877us (19072us total) trans 4294970061 rc 0/0 00000100:00100000:3.0:1788572538.073582:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3664 00000020:00080000:3.0:1788572538.073588:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572538.077923:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.080921:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360219776 00000100:00100000:3.0:1788572538.080931:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360219776 00000100:00100000:3.0:1788572538.080998:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3665 00000100:00100000:3.0:1788572538.081033:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35877:x1875453360219776:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.081111:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160f6b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x64:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081128:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160f6b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x64:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081138:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb160f6b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x64:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081461:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160f72 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x69:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081483:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb160d5e lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x69:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef73 expref: 67 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081492:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb160f72 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x69:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081522:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb160d5e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x69:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef73 expref: 67 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.081537:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb160d5e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x69:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eef73 expref: 67 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.081551:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5968380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382326272:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.081620:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb59c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788572538.088102:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572538.088113:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb160f79 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.088120:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572538.088125:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb160f79 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.088132:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb160f79 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.088147:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6a:0x0].0x0 (ffff9627d0ee2a00) 00010000:00010000:3.0:1788572538.088151:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572538.088156:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb160f79 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.088163:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb160f79 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.088181:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.088183:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.088185:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572538.089136:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.089196:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.131597:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360220800 00000100:00100000:3.0:1788572538.131603:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360220800 00010000:00010000:3.0:1788572538.131636:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb160e45 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x64:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eeff1 expref: 162 pid: 18263 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.131667:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 876 00000100:00100000:3.0:1788572538.131675:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:2349:x1875453360220800:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.131688:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.131696:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb160e45 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x64:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eeff1 expref: 162 pid: 18263 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.131708:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb160e45 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x64:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eeff1 expref: 162 pid: 18263 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.131729:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x64:0x0].0x0 (ffff9627c29da100) 00010000:00010000:3.0:1788572538.131734:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb160f8e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x64:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.131745:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb160f8e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x64:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.131753:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb160f8e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x64:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.131775:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.131812:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:2349:x1875453360220800:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 138us (852us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.131819:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 876 00010000:00010000:1.0:1788572538.132945:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb160f87 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.132957:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6b:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572538.132959:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.132963:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb160f87 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.132969:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb160f87 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133029:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133036:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133041:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133046:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.133049:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133054:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133062:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7b:0x0].0x0 (ffff9627e7a95c00) 00010000:00010000:1.0:1788572538.133064:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.133068:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133074:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb15e97e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133102:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133111:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133119:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133126:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133132:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133138:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133143:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.133146:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133151:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133158:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7b:0x0].0x0 (ffff9627e7a95c00) 00010000:00010000:1.0:1788572538.133160:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.133696:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133705:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f95 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133718:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133724:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133729:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133734:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.133736:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133742:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133750:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7b:0x0].0x0 (ffff9627e7a95c00) 00010000:00010000:1.0:1788572538.133753:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572538.133755:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.133761:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb160f80 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.133973:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35879:x1875453360220288:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 44593us (44840us total) trans 4294970073 rc 0/0 00000100:00100000:1.0:1788572538.133983:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3666 00000020:00080000:1.0:1788572538.133987:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572538.142924:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.143066:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360221056 00000100:00100000:1.0:1788572538.143077:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360221056 00000100:00100000:1.0:1788572538.143129:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 830 00000100:00100000:1.0:1788572538.143139:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35877:x1875453360221056:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572538.143545:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x69:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572538.143586:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970074 00010000:00010000:1.0:1788572538.143639:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x69:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.143658:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.143670:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.143679:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.143686:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.143692:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.143713:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572538.143815:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35877:x1875453360221056:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 675us (899us total) trans 4294970075 rc 0/0 00000100:00100000:1.0:1788572538.143823:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 830 00000020:00080000:1.0:1788572538.143827:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572538.145054:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572538.145065:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.145075:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572538.145080:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.145088:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.145103:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x69:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:3.0:1788572538.145106:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572538.145112:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.145118:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb160f9c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x69:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.145139:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.145141:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.145142:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572538.146562:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.146610:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572538.148101:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572538.148108:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb160fa3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.148115:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572538.148120:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb160fa3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.148128:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb160fa3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.148141:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6b:0x0].0x0 (ffff9627e7a95400) 00010000:00010000:3.0:1788572538.148144:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572538.148149:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb160fa3 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.148155:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb160fa3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.148172:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.148174:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.148176:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572538.167287:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.167318:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.167876:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.180609:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360222080 00000100:00100000:1.0:1788572538.180620:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360222080 00000100:00100000:1.0:1788572538.180683:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360222208 00000100:00100000:1.0:1788572538.180686:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360222208 00000100:00100000:1.0:1788572538.180708:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3670 00000100:00100000:1.0:1788572538.180717:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35883:x1875453360222080:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.180803:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb160fb1 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.185776:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb160fb1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.185793:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb160fb1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.190607:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360222592 00000100:00100000:1.0:1788572538.190618:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360222592 00000100:00100000:3.0:1788572538.190722:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.190825:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.191448:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360222848 00000100:00100000:3.0:1788572538.191454:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360222848 00000100:00100000:3.0:1788572538.191504:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 832 00000100:00100000:3.0:1788572538.191514:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+161:35867:x1875453360222848:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572538.192781:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:35867:x1875453360222848:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1258us (1945us total) trans 4294970078 rc 0/0 00000100:00100000:3.0:1788572538.192793:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 832 00000100:00100000:1.0:1788572538.193448:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360222336 00000100:00100000:1.0:1788572538.193456:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360222336 00000100:00100000:1.0:1788572538.193518:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3672 00000100:00100000:1.0:1788572538.193527:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35882:x1875453360222592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.193543:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.193562:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb160fdb lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef092 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.193664:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3673 00000100:00100000:3.0:1788572538.193675:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35884:x1875453360222336:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.193741:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193756:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193768:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.193779:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x80:0x0]: returning size 3 00010000:00010000:1.0:1788572538.193859:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193867:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193874:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193879:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.193884:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193892:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193905:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572538.193908:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572538.193913:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193920:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb160fe2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x80:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193956:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb160fdb lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef092 expref: 83 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193967:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572538.193970:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572538.193974:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb160fdb lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef092 expref: 83 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.193987:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:1.0:1788572538.194047:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35882:x1875453360222592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 519us (4011us total) trans 4294970079 rc 301/301 00000100:00100000:1.0:1788572538.194054:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3672 00010000:00010000:3.0:1788572538.194533:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160ff0 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.194546:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160ff0 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.194555:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb160ff0 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.194570:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160ff7 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x45fc7e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.194578:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160ff7 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x45fc7e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.194584:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb160ff7 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x45fc7e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.206705:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.206750:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.206810:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.209840:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360223104 00000100:00100000:1.0:1788572538.209858:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360223104 00000100:00100000:3.0:1788572538.210088:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360223232 00000100:00100000:3.0:1788572538.210094:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360223232 00000100:00100000:3.0:1788572538.210122:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360223488 00000100:00100000:3.0:1788572538.210125:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360223488 00000100:00100000:3.0:1788572538.210156:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3674 00000100:00100000:3.0:1788572538.210179:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35883:x1875453360223104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.210204:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.210226:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161005 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef099 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210339:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210355:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210367:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572538.210380:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7c:0x0]: returning size 3 00010000:00010000:3.0:1788572538.210526:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210534:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210541:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210547:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.210552:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210560:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210572:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7c:0x0].0x0 (ffff9627d599a600) 00010000:00010000:3.0:1788572538.210575:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572538.210580:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210587:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16100c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210631:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161005 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef099 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210642:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572538.210644:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572538.210648:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161005 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef099 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210662:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:3.0:1788572538.210724:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35883:x1875453360223104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 559us (4030us total) trans 4294970080 rc 301/301 00000100:00100000:3.0:1788572538.210732:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3674 00000100:00100000:3.0:1788572538.210752:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3675 00000100:00100000:3.0:1788572538.210759:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35882:x1875453360223232:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572538.210776:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x80:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572538.210843:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35882:x1875453360223232:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 84us (4091us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.210849:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3675 00000100:00100000:3.0:1788572538.210866:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3676 00000100:00100000:3.0:1788572538.210873:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:35867:x1875453360223488:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572538.210881:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.210891:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16101a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef110 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210959:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161021 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x63:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210968:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161021 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x63:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.210977:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161021 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x63:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572538.210990:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x63:0x0]: returning size 3 00010000:00010000:3.0:1788572538.211073:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16101a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef110 expref: 167 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.211087:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161021 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x63:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef110 expref: 166 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.211094:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x63:0x0].0x0 (ffff9627d0ee2400) 00010000:00010000:3.0:1788572538.211096:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572538.211099:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:3.0:1788572538.211121:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360223488:12345-192.168.202.40@tcp:101:diff.0 Request processed in 248us (4311us total) trans 4294970081 rc 0/0 00000100:00100000:3.0:1788572538.211128:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3676 00000100:00100000:3.0:1788572538.213540:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.223837:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360224000 00000100:00100000:1.0:1788572538.223849:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360224000 00000100:00100000:1.0:1788572538.223913:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3678 00000100:00100000:1.0:1788572538.223923:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35882:x1875453360224000:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.223957:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.223981:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161059 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef125 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.230633:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.230676:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.230713:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.230747:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.231718:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360224384 00000100:00100000:3.0:1788572538.231723:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360224384 00002000:00100000:3.0:1788572538.231758:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x38:0x0] (0->4095) req@ffff9627f5b65180 x1875453360224384/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:199/0 lens 488/0 e 0 to 0 dl 1788572549 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572538.231767:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5b65180 with x1875453360224384 ext(0->4095) 00002000:00010000:3.0:1788572538.231771:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f5b65180 00000100:00100000:3.0:1788572538.231803:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 215 00000100:00100000:3.0:1788572538.231810:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b65180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+31:2347:x1875453360224384:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572538.232043:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b65180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+31:2347:x1875453360224384:12345-192.168.202.40@tcp:3:diff.0 Request processed in 232us (1327us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572538.232050:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 215 00002000:00100000:3.0:1788572538.232058:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x38:0x0] (0->4095) req@ffff9627f5b65180 x1875453360224384/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:199/0 lens 488/4536 e 0 to 0 dl 1788572549 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572538.232066:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5b65180 with x1875453360224384 ext(0->4095) 00002000:00010000:3.0:1788572538.232070:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f5b65180 00000100:00100000:1.0:1788572538.239400:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.247594:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35882:x1875453360224768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.247616:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.247620:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572538.247780:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16106e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef125 expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.247802:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572538.247807:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572538.247893:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161091 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef12c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.248693:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360225152 00000100:00100000:1.0:1788572538.248699:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360225152 00000100:00100000:1.0:1788572538.248741:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 833 00000100:00100000:1.0:1788572538.248750:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+163:35867:x1875453360225152:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572538.249079:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:35867:x1875453360225152:12345-192.168.202.40@tcp:35:diff.0 Request processed in 329us (468us total) trans 4294970083 rc 0/0 00000100:00100000:1.0:1788572538.249091:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 833 00000004:00080000:3.0:1788572538.249210:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x3d:0x0], index 1: 0 00010000:00010000:3.0:1788572538.249455:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161098 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.249481:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161098 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.249492:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161098 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.249552:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161091 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef12c expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.249574:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161098 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef12c expref: 86 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.249582:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d599a500) 00010000:00010000:3.0:1788572538.249585:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572538.249588:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52c00, rc 0) 00000100:00100000:3.0:1788572538.251202:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35882:x1875453360224768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3618us (4154us total) trans 4294970084 rc 0/0 00000100:00100000:3.0:1788572538.251213:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3682 00000100:00100000:1.0:1788572538.255799:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360225536 00000100:00100000:1.0:1788572538.255806:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360225536 00000100:00100000:1.0:1788572538.255853:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 234 00000100:00100000:1.0:1788572538.255862:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:2347:x1875453360225536:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572538.260147:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.260177:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1610bb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef13a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260251:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260265:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260277:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260290:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260296:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260302:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260483:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260495:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260500:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260505:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.260510:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260517:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260529:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0x0 (ffff9627d20cbd00) 00010000:00010000:1.0:1788572538.260532:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.260537:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260543:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610c2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260553:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260558:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260563:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260567:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.260570:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260575:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260581:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0xa16187a3 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572538.260583:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.260585:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260590:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610c9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x60:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260619:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1610bb lrc: 1/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef13a expref: 168 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260741:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0x0 (ffff9627d20cbd00) 00010000:00010000:1.0:1788572538.260744:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572538.260748:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1610bb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef13a expref: 168 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.260758:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc -2) 00000100:00100000:3.0:1788572538.271141:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.271539:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.273710:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360225792 00000100:00100000:1.0:1788572538.273727:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360225792 00000100:00100000:1.0:1788572538.277161:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 297 00000100:00100000:1.0:1788572538.277185:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:2347:x1875453360225792:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572538.277205:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572538.277253:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1610ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0x62:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef14f expref: 166 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.277277:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x62:0x0].0x0 (ffff9627d0ee2600) 00010000:00010000:1.0:1788572538.277282:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572538.297676:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:2347:x1875453360225792:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 13792us (19835us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.297708:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 297 00000100:00100000:1.0:1788572538.311937:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360226432 00000100:00100000:1.0:1788572538.311953:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360226432 00000100:00100000:1.0:1788572538.312034:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360226688 00000100:00100000:1.0:1788572538.312038:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360226688 00000100:00100000:1.0:1788572538.312063:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3685 00000100:00100000:1.0:1788572538.312073:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35882:x1875453360226432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.312094:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.312120:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1610de lrc: 2/0,0 mode: --/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef228 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312157:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312171:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312183:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312294:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1610de lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef228 expref: 86 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312320:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef228 expref: 85 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312329:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572538.312332:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572538.312336:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc 0) 00000100:00100000:1.0:1788572538.312392:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35882:x1875453360226432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 320us (611us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.312399:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3685 00000100:00100000:1.0:1788572538.312416:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3686 00000100:00100000:1.0:1788572538.312423:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360226688:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572538.312435:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.312445:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef22f expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.312523:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00010000:00010000:1.0:1788572538.312647:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610ec lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef22f expref: 168 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312659:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572538.312662:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572538.312665:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1610ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef22f expref: 168 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.312675:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e800, rc 0) 00000100:00100000:1.0:1788572538.312704:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360226688:12345-192.168.202.40@tcp:101:diff.0 Request processed in 281us (864us total) trans 4294970087 rc 301/301 00000100:00100000:1.0:1788572538.312710:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3686 00000100:00100000:3.0:1788572538.319579:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360226944 00000100:00100000:3.0:1788572538.319593:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360226944 00000100:00100000:3.0:1788572538.319652:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3687 00000100:00100000:3.0:1788572538.319664:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35883:x1875453360226944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.319685:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.319708:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1610fa lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef236 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572538.320323:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x3e:0x0], index 0: 0 00010000:00010000:3.0:1788572538.320567:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161101 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.320584:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161101 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.320600:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161101 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.320659:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1610fa lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef236 expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.320678:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161101 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef236 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.320687:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627d599a400) 00010000:00010000:3.0:1788572538.320690:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572538.320693:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:3.0:1788572538.320756:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35883:x1875453360226944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1094us (1676us total) trans 4294970088 rc 0/0 00000100:00100000:3.0:1788572538.320764:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3687 00000100:00100000:1.0:1788572538.335370:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360227200 00000100:00100000:1.0:1788572538.335381:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360227200 00000100:00100000:1.0:1788572538.335442:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3688 00000100:00100000:1.0:1788572538.335453:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35882:x1875453360227200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.335562:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16110f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.335579:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16110f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.335591:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16110f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572538.335992:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].47, 88 off11920 00000004:00080000:1.0:1788572538.336045:0:22536:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 886405040092ns, next at 887404996917ns, rc = 0 00000004:00010000:1.0:1788572538.336142:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a997d20f reply state 00000000b2e96bfa transno 4294970089 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16110f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.336152:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572538.336253:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35882:x1875453360227200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 800us (1035us total) trans 4294970089 rc 0/0 00000100:00100000:1.0:1788572538.336260:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3688 00000100:00100000:3.0:1788572538.336548:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360227456 00000100:00100000:3.0:1788572538.336555:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360227456 00000100:00100000:3.0:1788572538.336598:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 835 00000100:00100000:3.0:1788572538.336606:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360227456:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572538.336744:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:35867:x1875453360227456:12345-192.168.202.40@tcp:35:diff.0 Request processed in 139us (1458us total) trans 4294970090 rc 0/0 00000100:00100000:3.0:1788572538.336751:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 835 00000100:00100000:3.0:1788572538.342266:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360227712 00000100:00100000:3.0:1788572538.342278:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360227712 00000100:00100000:3.0:1788572538.342339:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3689 00000100:00100000:3.0:1788572538.342350:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35883:x1875453360227712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.342374:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.342394:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161116 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef23d expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.342465:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.342493:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.342724:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.342744:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.342753:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.342760:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343356:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16112b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343367:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16112b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343374:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16112b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572538.343401:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:3.0:1788572538.343405:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572538.343423:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16112b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343435:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343442:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343449:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343454:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.343457:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343463:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343483:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572538.343486:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.343490:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343652:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16111d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343670:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343676:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343686:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343691:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.343694:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343700:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343708:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d599a100) 00010000:00010000:3.0:1788572538.343711:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.343714:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343720:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161124 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343755:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161116 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef23d expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343767:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16112b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef23d expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.343774:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572538.343776:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572538.343779:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:3.0:1788572538.343892:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35883:x1875453360227712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1543us (2050us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.343903:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3689 00000100:00100000:3.0:1788572538.346834:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360227840 00000100:00100000:3.0:1788572538.346856:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360227840 00000100:00100000:3.0:1788572538.346924:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3690 00000100:00100000:3.0:1788572538.346934:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35884:x1875453360227840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.346969:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.346992:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161132 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef244 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572538.347428:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x3e:0x0], index 1: 0 00010000:00010000:3.0:1788572538.347658:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161139 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.347672:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161139 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.347682:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161139 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.347740:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161132 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef244 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.347759:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161139 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef244 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.347767:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a300) 00010000:00010000:3.0:1788572538.347770:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572538.347774:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:3.0:1788572538.347828:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35884:x1875453360227840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 896us (1090us total) trans 4294970091 rc 0/0 00000100:00100000:3.0:1788572538.347837:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3690 00000100:00100000:3.0:1788572538.358592:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.358671:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.358692:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.358709:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360228096 00000100:00100000:3.0:1788572538.358716:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360228096 00000100:00100000:3.0:1788572538.358748:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360228352 00000100:00100000:3.0:1788572538.358751:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360228352 00000100:00100000:3.0:1788572538.358766:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360228480 00000100:00100000:3.0:1788572538.358768:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360228480 00000100:00100000:3.0:1788572538.358781:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3692 00000100:00100000:3.0:1788572538.358788:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:35867:x1875453360228096:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572538.358804:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.358818:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16114e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef252 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.359659:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.359678:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.359689:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.359706:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.359719:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.359725:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360374:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360386:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360394:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572538.360428:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc2:0x0]: returning size 3 00000004:00200000:3.0:1788572538.360432:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572538.360451:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360463:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360483:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360490:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.360497:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.360710:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3693 00000100:00100000:1.0:1788572538.360725:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35884:x1875453360228352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.360744:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.360757:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16116a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef259 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.360810:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.360820:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.360827:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.360839:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.360858:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.360866:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361108:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361119:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361126:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.361146:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:1.0:1788572538.361149:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572538.361162:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361172:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361177:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361183:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361188:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.361193:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361200:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361212:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572538.361216:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.361220:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361492:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.361496:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361511:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361522:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572538.361525:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.361528:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361535:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161155 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361550:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361556:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361562:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361567:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.361570:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361576:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361587:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x9beb5ca3 (ffff9627d599a900) 00010000:00010000:3.0:1788572538.361589:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.361592:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361597:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16115c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361631:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16114e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 170 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361644:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 169 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361651:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:3.0:1788572538.361654:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572538.361656:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c2477200, rc 0) 00000100:00100000:3.0:1788572538.361717:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360228096:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2928us (3125us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.361725:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3692 00000100:00100000:3.0:1788572538.361758:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3694 00000100:00100000:3.0:1788572538.361768:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35883:x1875453360228480:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.361803:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572538.361818:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161101 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef236 expref: 94 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.361834:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.361928:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161171 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361948:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361956:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361962:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361967:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.361971:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361977:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361985:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572538.361988:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.361991:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.361996:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161178 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.362042:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16116a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.362053:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 93 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.362059:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572538.362061:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572538.362064:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:1.0:1788572538.362090:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35884:x1875453360228352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1371us (3419us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.362096:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3693 00010000:00010000:3.0:1788572538.364186:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16112b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef23d expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.364213:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572538.364218:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572538.364592:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161186 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.364610:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.364620:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161186 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.364665:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.364681:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 91 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.364697:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382327552:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.364790:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f14e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.364928:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.374133:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360228608 00000100:00100000:1.0:1788572538.374151:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360228608 00000100:00100000:3.0:1788572538.374317:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 910 00000100:00100000:3.0:1788572538.374329:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360228608:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572538.374668:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360228608:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 340us (9739us total) trans 4294967521 rc 0/0 00000100:00100000:3.0:1788572538.374677:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 910 00000100:00100000:3.0:1788572538.374791:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382327552:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.374817:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161186 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.381690:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360228864 00000100:00100000:1.0:1788572538.381701:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360228864 00000100:00100000:1.0:1788572538.381769:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3695 00000100:00100000:1.0:1788572538.381780:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360228864:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572538.381805:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.381829:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16118d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef260 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.381959:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.381976:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.381988:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.382043:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc2:0x0]: returning size 3 00010000:00010000:1.0:1788572538.382159:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382167:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382174:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382180:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.382185:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382198:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382213:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:1.0:1788572538.382216:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572538.382222:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382243:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161194 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382291:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16118d lrc: 1/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef260 expref: 171 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382302:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572538.382305:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572538.382309:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16118d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef260 expref: 171 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.382319:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:1.0:1788572538.382386:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360228864:12345-192.168.202.40@tcp:101:diff.0 Request processed in 608us (1068us total) trans 4294970092 rc 301/301 00000100:00100000:1.0:1788572538.382394:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3695 00000100:00100000:1.0:1788572538.395303:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.395463:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360228992 00000100:00100000:3.0:1788572538.395484:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360228992 00010000:00010000:3.0:1788572538.395543:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 92 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.395591:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 877 00000100:00100000:3.0:1788572538.395604:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:2350:x1875453360228992:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.395620:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.395627:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 92 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.395641:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 92 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.395665:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16117f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef259 expref: 92 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.395682:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572538.395689:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161186 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.395701:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161186 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.395709:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161186 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.395748:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.395813:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:2350:x1875453360228992:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 211us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.395822:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 877 00000100:00100000:1.0:1788572538.400077:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.400123:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.401254:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360229376 00000100:00100000:3.0:1788572538.401260:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360229376 00000100:00100000:3.0:1788572538.401304:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3696 00000100:00100000:3.0:1788572538.401312:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360229376:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572538.401330:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.401350:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef267 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.401399:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.401411:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 4/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 171 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.401430:0:5837:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.401443:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 3/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 171 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.401457:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:3.0:1788572538.401461:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572538.401465:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 3/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 171 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.401484:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e76eb400, rc 0) 00000100:00100000:3.0:1788572538.401532:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360229376:12345-192.168.202.40@tcp:101:diff.0 Request processed in 220us (1407us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.401539:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3696 00000100:00100000:1.0:1788572538.411517:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.411559:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.411600:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.418067:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.418104:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.425693:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360230528 00000100:00100000:3.0:1788572538.425703:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360230528 00000100:00100000:3.0:1788572538.425764:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3702 00000100:00100000:3.0:1788572538.425775:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35884:x1875453360230528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.425800:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.425837:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb1611e1 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef27c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.425890:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.425903:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.425914:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.426052:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb1611e1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.426069:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 94 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.426077:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:3.0:1788572538.426080:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572538.426083:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e76ea200, rc 0) 00000100:00100000:3.0:1788572538.426141:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35884:x1875453360230528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 367us (780us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.426147:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3702 00000100:00100000:1.0:1788572538.431278:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360230912 00000100:00100000:1.0:1788572538.431294:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360230912 00000100:00100000:1.0:1788572538.431358:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3703 00000100:00100000:1.0:1788572538.431369:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35884:x1875453360230912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.431465:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.431497:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.431511:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572538.431908:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].48, 88 off12008 00000004:00010000:1.0:1788572538.431994:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006e9c1e1b reply state 000000001bb052cf transno 4294970095 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.432044:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572538.432137:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35884:x1875453360230912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 769us (2389us total) trans 4294970095 rc 0/0 00000100:00100000:1.0:1788572538.432144:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3703 00010000:00010000:1.0:1788572538.440022:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.440035:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.440043:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.440050:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.440055:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.440063:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.440080:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.440082:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572538.440088:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 2/0,0 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.440094:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1611ef lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.447148:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360231040 00000100:00100000:3.0:1788572538.447167:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360231040 00000100:00100000:1.0:1788572538.447559:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360231168 00000100:00100000:1.0:1788572538.447565:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360231168 00000100:00100000:1.0:1788572538.447632:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3704 00000100:00100000:1.0:1788572538.447643:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35883:x1875453360231168:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.447670:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.447694:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef291 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.447770:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.447788:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.448034:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:1.0:1788572538.448039:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.448044:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:1.0:1788572538.448118:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35883:x1875453360231168:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 475us (8159us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.448125:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3704 00000100:00100000:1.0:1788572538.451526:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360231424 00000100:00100000:1.0:1788572538.451541:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360231424 00000100:00100000:1.0:1788572538.451602:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3705 00000100:00100000:1.0:1788572538.451613:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35882:x1875453360231424:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.451755:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35882:x1875453360231424:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 143us (3474us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.451764:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3705 00000100:00100000:3.0:1788572538.452514:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.454041:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360231552 00000100:00100000:1.0:1788572538.454052:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360231552 00000100:00100000:1.0:1788572538.454102:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 912 00000100:00100000:1.0:1788572538.454111:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360231552:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.454408:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360231552:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 297us (1896us total) trans 4294967523 rc 0/0 00000100:00100000:1.0:1788572538.454415:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 912 00000100:00100000:3.0:1788572538.461978:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360231680 00000100:00100000:3.0:1788572538.461990:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360231680 00000100:00100000:3.0:1788572538.462080:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360231808 00000100:00100000:3.0:1788572538.462084:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360231808 00000100:00100000:3.0:1788572538.462112:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 913 00000100:00100000:3.0:1788572538.462121:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360231680:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572538.462415:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360231680:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 295us (874us total) trans 4294967522 rc 0/0 00000100:00100000:3.0:1788572538.462422:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 913 00000100:00100000:3.0:1788572538.462438:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 914 00000100:00100000:3.0:1788572538.462442:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35882:x1875453360231808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.462449:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.462462:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef298 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.462501:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.462512:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.462516:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.462522:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 4/0,0 mode: --/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.462555:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.462569:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.462584:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382327680:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.463666:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f376c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.476589:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382327680:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.476630:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.476652:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e76eba00, rc 0) 00000100:00100000:3.0:1788572538.476711:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35882:x1875453360231808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 14268us (15114us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.476720:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 914 00000100:00100000:3.0:1788572538.476941:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360232704 00000100:00100000:3.0:1788572538.476957:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360232704 00010000:00010000:3.0:1788572538.476987:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.477036:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 878 00000100:00100000:3.0:1788572538.477046:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360232704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.477060:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.477083:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.477095:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.477117:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a68d18f4/0x85ace67feb1611f6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef283 expref: 14 pid: 6681 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.477132:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.477137:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.477145:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.477155:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.477189:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.477205:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382327808:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572538.477257:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f376c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.481324:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360232320 00000100:00100000:1.0:1788572538.481335:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360232320 00000100:00100000:1.0:1788572538.481397:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3707 00000100:00100000:1.0:1788572538.481406:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35883:x1875453360232320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.481499:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35883:x1875453360232320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 93us (5300us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.481507:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3707 00000100:00100000:3.0:1788572538.490532:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382327808:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572538.490563:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.490643:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360232704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 13597us (13841us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.490655:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 878 00000100:00100000:3.0:1788572538.505145:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360233088 00000100:00100000:3.0:1788572538.505150:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360233088 00002000:00100000:3.0:1788572538.505203:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x3d:0x0] (0->2) req@ffff9626c36dea00 x1875453360233088/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:199/0 lens 496/0 e 0 to 0 dl 1788572549 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572538.505227:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36dea00 with x1875453360233088 ext(0->2) 00002000:00010000:3.0:1788572538.505234:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c36dea00 00000100:00100000:3.0:1788572538.505250:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 217 00000100:00100000:3.0:1788572538.505256:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360233088:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572538.505791:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360233088:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 534us (1325us total) trans 4294967523 rc 0/0 00000100:00100000:3.0:1788572538.505799:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 217 00002000:00100000:3.0:1788572538.505806:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x3d:0x0] (0->2) req@ffff9626c36dea00 x1875453360233088/t4294967523(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:199/0 lens 496/448 e 0 to 0 dl 1788572549 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572538.506237:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36dea00 with x1875453360233088 ext(0->2) 00002000:00010000:3.0:1788572538.506242:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c36dea00 00000100:00100000:1.0:1788572538.527556:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360233344 00000100:00100000:1.0:1788572538.527567:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360233344 00000100:00100000:1.0:1788572538.527635:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3710 00000100:00100000:1.0:1788572538.527645:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35884:x1875453360233344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.527667:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.527688:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161220 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2a6 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.527759:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161227 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.527773:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161227 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.527786:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161227 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.527833:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572538.527852:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161227 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.527880:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161220 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2a6 expref: 102 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.527898:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161227 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2a6 expref: 101 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.527912:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.527915:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572538.527919:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:1.0:1788572538.527996:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35884:x1875453360233344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 353us (595us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.528035:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3710 00000100:00100000:1.0:1788572538.534725:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360233728 00000100:00100000:1.0:1788572538.534736:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360233728 00000100:00100000:1.0:1788572538.534797:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 916 00000100:00100000:1.0:1788572538.534808:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360233728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.534826:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.534845:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2b4 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572538.534871:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7379, blocks: 0x1 00010000:00010000:1.0:1788572538.534891:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3e:0x0].0x0 since=886s v=(0 0) 00010000:00010000:1.0:1788572538.534896:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572538.534904:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2b4 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.534922:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2b4 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.534935:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:1.0:1788572538.535029:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360233728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 222us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.535037:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 916 00000100:00100000:1.0:1788572538.550104:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.550149:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360234112 00000100:00100000:1.0:1788572538.550158:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360234112 00000100:00100000:1.0:1788572538.550218:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3712 00000100:00100000:1.0:1788572538.550241:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35883:x1875453360234112:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.550348:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35883:x1875453360234112:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 110us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.550356:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3712 00000100:00100000:1.0:1788572538.556356:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360234240 00000100:00100000:1.0:1788572538.556369:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360234240 00000100:00100000:1.0:1788572538.556435:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3713 00000100:00100000:1.0:1788572538.556444:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35884:x1875453360234240:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.556695:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35884:x1875453360234240:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 250us (1234us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.556702:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3713 00000100:00100000:3.0:1788572538.565363:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360234368 00000100:00100000:3.0:1788572538.565373:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360234368 00000100:00100000:3.0:1788572538.565418:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360234496 00000100:00100000:3.0:1788572538.565421:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360234496 00000100:00100000:3.0:1788572538.565439:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 917 00000100:00100000:3.0:1788572538.565448:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35883:x1875453360234368:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.565467:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.565499:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2bb expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565534:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16122e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ad expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565544:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.565549:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.565555:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 4/0,0 mode: --/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565592:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16122e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ad expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565605:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16122e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ad expref: 14 pid: 6682 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.565619:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382327936:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.565731:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f376c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.565752:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 918 00000100:00100000:3.0:1788572538.565760:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35884:x1875453360234496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.565769:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.565780:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f61d707e/0x85ace67feb161243 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2c2 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565797:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2b4 expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565805:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.565808:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.565812:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f61d707e/0x85ace67feb161243 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2c2 expref: 17 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565832:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2b4 expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.565842:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2b4 expref: 17 pid: 20911 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.565851:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5edc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382328064:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.565875:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.575322:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 880 00000100:00100000:3.0:1788572538.575333:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360234624:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.575342:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.575358:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16122e lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ad expref: 15 pid: 6682 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.575370:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16122e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ad expref: 15 pid: 6682 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.575385:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.575388:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572538.575393:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.575402:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.575424:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.575434:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382328320:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572538.575522:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c1254300 going to sleep for 11 seconds 00010000:00010000:3.0:1788572538.575560:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16122e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ad expref: 15 pid: 6682 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.575576:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382327936:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.575606:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.575617:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee600, rc 0) 00000100:00100000:3.0:1788572538.575662:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35883:x1875453360234368:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 10214us (10430us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.575669:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 917 00010000:00010000:1.0:1788572538.576887:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c7db23f9/0x85ace67feb161235 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2b4 expref: 18 pid: 20911 timeout: 986 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.576904:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5edc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382328064:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572538.576927:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f61d707e/0x85ace67feb161243 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2c2 expref: 17 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.576956:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc 0) 00000100:00100000:1.0:1788572538.576989:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35884:x1875453360234496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 11230us (11626us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.576996:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 918 00000100:00100000:3.0:1788572538.611338:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.611388:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.611429:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382328320:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572538.611454:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.612518:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360234624:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 37186us (37716us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.612526:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 880 00000100:00100000:1.0:1788572538.647986:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360235136 00000100:00100000:1.0:1788572538.647998:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360235136 00000100:00100000:1.0:1788572538.648077:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360235264 00000100:00100000:1.0:1788572538.648081:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360235264 00000100:00100000:1.0:1788572538.648105:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3714 00000100:00100000:1.0:1788572538.648115:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35884:x1875453360235136:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.648365:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35884:x1875453360235136:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 249us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.648375:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3714 00000100:00100000:1.0:1788572538.648395:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3715 00000100:00100000:1.0:1788572538.648402:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35883:x1875453360235264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572538.648446:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35883:x1875453360235264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 44us (628us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.648452:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3715 00000100:00100000:1.0:1788572538.651699:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.657406:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360235520 00000100:00100000:1.0:1788572538.657426:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360235520 00000100:00100000:1.0:1788572538.657497:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 920 00000100:00100000:1.0:1788572538.657510:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360235520:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1788572538.685619:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970094 is committed 00000004:00080000:1.0:1788572538.685640:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 37 committed 00000020:00080000:1.0:1788572538.685670:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 84 batchid updated 00000001:00080000:1.0:1788572538.685685:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970051 is committed 00000001:00080000:1.0:1788572538.685692:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970095 is committed 00000001:00080000:1.0:1788572538.685698:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970092 is committed 00000004:00080000:1.0:1788572538.685702:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 32 committed 00000020:00080000:1.0:1788572538.685727:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627dffb3cc0 refcount 2 committed 0 result 0 batchid 87 00000020:00080000:1.0:1788572538.685732:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de100 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627dffb36c0 00000020:00080000:1.0:1788572538.685735:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff03d400 00000001:00080000:1.0:1788572538.685747:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970097 is committed 00000020:00080000:1.0:1788572538.685769:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ec075f00 refcount 2 committed 0 result 0 batchid 86 00000020:00080000:1.0:1788572538.685771:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceac00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d2241e40 00000020:00080000:1.0:1788572538.685774:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cead00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627e7420400 00000020:00080000:1.0:1788572538.685811:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2eb56c0 refcount 2 committed 0 result 0 batchid 85 00000020:00080000:1.0:1788572538.685815:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad300 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2eb5f00 00000020:00080000:1.0:1788572538.685818:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8adb00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627e7423000 00000100:00100000:1.0:1788572538.685982:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382328576:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572538.686353:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572538.686413:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788572538.686420:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382328704:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572538.686443:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572538.686483:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382328576 00000100:00100000:1.0:1788572538.686491:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382328576 00000100:00100000:3.0:1788572538.686493:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788572538.686527:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382328704 00000100:00100000:1.0:1788572538.686530:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382328704 00000100:00100000:1.0:1788572538.686550:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 921 00000100:00100000:1.0:1788572538.686559:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3302:x1875453382328576:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572538.686586:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382328448:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572538.686616:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572538.686636:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382328448 00000100:00100000:3.0:1788572538.686640:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382328448 00000100:00100000:3.0:1788572538.686660:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 922 00000100:00100000:3.0:1788572538.686667:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382328704:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1788572538.686838:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572538.686842:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb mtime from disk: 0 -> 1788572538 00002000:00010000:3.0:1788572538.686845:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1788572538.686858:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb ctime from disk: 1788572537 -> 1788572538 00002000:00010000:3.0:1788572538.686861:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572538.686897:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382328704:12345-0@lo:2:osp-syn-1-0.0 Request processed in 231us (454us total) trans 4294967525 rc 0/0 00000100:00100000:3.0:1788572538.686904:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 922 00000100:00100000:3.0:1788572538.686923:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 923 00000100:00100000:3.0:1788572538.686928:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7662050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382328448:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1788572538.686996:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x3a:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572538.687000:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x3a:0x0] updating lvb mtime from disk: 1788572522 -> 1788572522 00002000:00010000:3.0:1788572538.687022:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x3a:0x0] updating lvb atime from disk: 1788572522 -> 1788572522 00002000:00010000:3.0:1788572538.687024:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x3a:0x0] updating lvb ctime from disk: 1788572523 -> 1788572523 00002000:00010000:3.0:1788572538.687027:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x3a:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572538.687056:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7662050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382328448:12345-0@lo:2:osp-syn-1-0.0 Request processed in 127us (439us total) trans 4294967527 rc 0/0 00000100:00100000:3.0:1788572538.687063:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 923 00000004:00080000:3.0:1788572538.688354:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfdbdc00/2, rc 0, transno 4294967527 00000100:00100000:3.0:1788572538.688377:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382328448:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572538.688389:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7874e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382328832:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572538.688414:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572538.688426:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788572538.688431:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7874a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382328960:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572538.688450:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572538.688485:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382329216 00000100:00100000:3.0:1788572538.688490:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382329216 00000100:00100000:3.0:1788572538.688528:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382329088 00000100:00100000:3.0:1788572538.688531:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382329088 00000100:00100000:3.0:1788572538.688540:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382328832 00000100:00100000:3.0:1788572538.688542:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382328832 00000100:00100000:3.0:1788572538.688550:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382328960 00000100:00100000:3.0:1788572538.688552:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382328960 00000100:00100000:3.0:1788572538.688562:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 924 00000100:00100000:3.0:1788572538.688570:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382329216:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:3.0:1788572538.688732:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3e:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572538.688736:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3e:0x0] updating lvb mtime from disk: 0 -> 1788572538 00002000:00010000:3.0:1788572538.688739:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x3e:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1788572538.688741:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3e:0x0] updating lvb ctime from disk: 1788572537 -> 1788572538 00002000:00010000:3.0:1788572538.688744:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x3e:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572538.688789:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3303:x1875453382329216:12345-0@lo:2:osp-syn-0-0.0 Request processed in 219us (822us total) trans 4294967525 rc 0/0 00000100:00100000:3.0:1788572538.688796:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 924 00000100:00100000:3.0:1788572538.688812:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1788572538.688819:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382329088:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572538.688919:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3303:x1875453382329088:12345-0@lo:2:osp-syn-0-0.0 Request processed in 100us (870us total) trans 4294967526 rc 0/0 00000100:00100000:3.0:1788572538.688926:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1788572538.688938:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1788572538.688944:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382328832:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:3.0:1788572538.689034:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3b:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572538.689037:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3b:0x0] updating lvb mtime from disk: 1788572522 -> 1788572522 00002000:00010000:3.0:1788572538.689040:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x3b:0x0] updating lvb atime from disk: 1788572522 -> 1788572522 00002000:00010000:3.0:1788572538.689042:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3b:0x0] updating lvb ctime from disk: 1788572523 -> 1788572523 00002000:00010000:3.0:1788572538.689044:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x3b:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572538.689075:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382328832:12345-0@lo:2:osp-syn-0-0.0 Request processed in 131us (660us total) trans 4294967527 rc 0/0 00000100:00100000:1.0:1788572538.690510:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382328576:12345-0@lo:2:osp-syn-1-0.0 Request processed in 3949us (4155us total) trans 4294967526 rc 0/0 00000100:00100000:1.0:1788572538.690519:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 921 00000100:00100000:3.0:1788572538.690623:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1788572538.690646:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 927 00000100:00100000:3.0:1788572538.690657:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382328960:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572538.690698:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.690717:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360235648 00000100:00100000:1.0:1788572538.690723:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360235648 00000100:00100000:1.0:1788572538.690748:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 928 00000100:00100000:1.0:1788572538.690754:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2347:x1875453360235648:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00002000:00010000:3.0:1788572538.690785:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572538.690788:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb mtime from disk: 1788572522 -> 1788572522 00002000:00010000:3.0:1788572538.690790:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb atime from disk: 1788572522 -> 1788572522 00002000:00010000:3.0:1788572538.690793:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb ctime from disk: 1788572523 -> 1788572523 00002000:00010000:3.0:1788572538.690795:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x3c:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572538.690829:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382328960:12345-0@lo:2:osp-syn-0-0.0 Request processed in 176us (2377us total) trans 4294967528 rc 0/0 00000100:00100000:3.0:1788572538.690835:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 927 00000004:00080000:3.0:1788572538.690867:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7874a80/2, rc 0, transno 4294967528 00000100:00100000:3.0:1788572538.690872:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7874a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382328960:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1788572538.690882:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7874e00/2, rc 0, transno 4294967527 00000100:00100000:3.0:1788572538.690885:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7874e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382328832:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1788572538.691990:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfdbc380/2, rc 0, transno 4294967525 00000100:00100000:3.0:1788572538.692029:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382328704:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572538.692043:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfdbf100/2, rc 0, transno 4294967526 00000100:00100000:3.0:1788572538.692047:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382328576:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572538.718385:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.718512:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360235904 00000100:00100000:3.0:1788572538.718523:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360235904 00000100:00100000:3.0:1788572538.718617:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3716 00000100:00100000:3.0:1788572538.718627:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35883:x1875453360235904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.718687:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.718713:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161219 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef29f expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.718731:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:3.0:1788572538.718736:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572538.718767:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.718786:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.718796:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572538.718942:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000efd66af5 reply state 00000000e1614a96 transno 4294970098 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.718952:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572538.719083:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35883:x1875453360235904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 456us (707us total) trans 4294970098 rc 0/0 00000100:00100000:3.0:1788572538.719090:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3716 00000100:00100000:3.0:1788572538.725034:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572538.725066:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.725079:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.725087:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.725094:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.725099:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.725108:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.725130:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:3.0:1788572538.725133:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.725138:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.725146:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16124a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.732044:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360236288 00000100:00100000:1.0:1788572538.732055:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360236288 00000100:00100000:1.0:1788572538.732158:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3717 00000100:00100000:1.0:1788572538.732169:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35883:x1875453360236288:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.732485:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161251 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.732504:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.732515:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161251 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.732552:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.732571:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 65 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.732586:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382329344:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572538.732686:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9be3300 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.738667:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360236416 00000100:00100000:1.0:1788572538.738677:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360236416 00010000:00010000:1.0:1788572538.738721:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 66 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.738764:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 881 00000100:00100000:1.0:1788572538.738775:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360236416:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572538.738790:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572538.738797:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 66 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.738811:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 66 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.738833:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb1611b7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef26e expref: 66 pid: 5839 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.738861:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:1.0:1788572538.738867:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161251 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.738883:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161251 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.738891:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161251 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.738913:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572538.739156:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360236416:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 382us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.739165:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 881 00000100:00100000:3.0:1788572538.751797:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.759746:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382329472:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.759779:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.770819:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360236928 00000100:00100000:1.0:1788572538.770832:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360236928 00000100:00100000:1.0:1788572538.770888:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3718 00000100:00100000:1.0:1788572538.770898:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35884:x1875453360236928:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.770970:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572538.771000:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161227 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2a6 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.771049:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.771055:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572538.771086:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.771100:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.771109:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572538.771478:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b34466b6 reply state 0000000062627668 transno 4294970099 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.771489:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572538.771597:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35884:x1875453360236928:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 699us (853us total) trans 4294970099 rc 0/0 00000100:00100000:1.0:1788572538.771605:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3718 00010000:00010000:1.0:1788572538.774236:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.774355:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.774364:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.774370:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.774375:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.774383:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.774400:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.774403:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.774407:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.774413:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16125f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.775883:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360236672 00000100:00100000:3.0:1788572538.775897:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360236672 00010000:00010000:3.0:1788572538.775964:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 12 pid: 6681 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.776341:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 882 00000100:00100000:3.0:1788572538.776353:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360236672:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.776371:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.776394:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 12 pid: 6681 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.776409:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 12 pid: 6681 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.776437:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb16123c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x3e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2bb expref: 12 pid: 6681 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.776453:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572538.776458:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572538.776464:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.776486:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.777395:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360237056 00000100:00100000:1.0:1788572538.777402:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360237056 00000100:00100000:1.0:1788572538.777439:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 931 00000100:00100000:1.0:1788572538.777447:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360237056:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572538.777720:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x3e:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572538.777724:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x3e:0x0] updating lvb mtime from disk: 0 -> 1788572520 00002000:00010000:1.0:1788572538.777726:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x3e:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:1.0:1788572538.777729:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x3e:0x0] updating lvb ctime from disk: 1788572537 -> 1788572524 00002000:00010000:1.0:1788572538.777731:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x3e:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572538.777787:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360237056:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 341us (813us total) trans 4294967528 rc 0/0 00000100:00100000:1.0:1788572538.777795:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 931 00010000:00010000:3.0:1788572538.778553:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.778588:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360236672:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 2236us (7904us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.778597:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 882 00010000:00010000:1.0:1788572538.782891:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.782912:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.782954:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.782963:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.782972:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.782979:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572538.782984:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.782993:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.783491:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.783512:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb161258 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.783623:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360236544:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 30946us (31832us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.783633:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 930 00000100:00100000:3.0:1788572538.799526:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360237312 00000100:00100000:3.0:1788572538.799542:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360237312 00000100:00100000:3.0:1788572538.799608:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360237568 00000100:00100000:3.0:1788572538.799612:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360237568 00000100:00100000:3.0:1788572538.799637:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3719 00000100:00100000:3.0:1788572538.799647:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35884:x1875453360237312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.799720:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161266 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.799740:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.799756:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161266 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.799793:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.799809:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 65 pid: 5837 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.799824:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c4700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382329600:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.799907:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.799927:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3720 00000100:00100000:3.0:1788572538.799935:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35883:x1875453360237568:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.799990:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16126d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800029:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16126d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800042:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16126d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800720:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161274 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800735:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 66 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800748:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161274 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800774:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 66 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.800783:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 66 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.800795:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382329728:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572538.800831:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.807887:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.807914:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.807966:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382329600:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.807994:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161266 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.808761:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360237824 00000100:00100000:3.0:1788572538.808769:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360237824 00010000:00010000:3.0:1788572538.808807:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 67 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.808808:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360237696 00000100:00100000:1.0:1788572538.808812:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360237696 00010000:00010000:1.0:1788572538.808826:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 68 pid: 5837 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.808843:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 883 00000100:00100000:3.0:1788572538.809486:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360237824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.809574:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.809583:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 68 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.809598:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1611fd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef291 expref: 68 pid: 5838 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.809621:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:3.0:1788572538.809627:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161274 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.809638:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161274 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.809645:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161274 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.809665:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.809735:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360237824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 268us (1858us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.811493:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 884 00000100:00100000:1.0:1788572538.811528:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0faf850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360237696:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572538.811551:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572538.811564:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 68 pid: 5837 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.811583:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 68 pid: 5837 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.811605:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb1611e8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef27c expref: 68 pid: 5837 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.811620:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.811626:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161266 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.811635:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161266 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.811642:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161266 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.811677:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572538.811738:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0faf850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360237696:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 232us (3822us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.811746:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 884 00000100:00100000:3.0:1788572538.811934:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 883 00000100:00100000:1.0:1788572538.818758:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.832974:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.836927:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572538.839751:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572538.839764:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161289 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.839778:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572538.839783:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161289 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.839791:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161289 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.839807:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6d:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:1.0:1788572538.839810:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572538.839815:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161289 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.839821:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161289 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.839841:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572538.839842:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572538.839844:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572538.840730:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360238592 00000100:00100000:1.0:1788572538.840739:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360238592 00000100:00100000:1.0:1788572538.840798:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3721 00000100:00100000:1.0:1788572538.840809:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35884:x1875453360238592:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.840981:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161290 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.840996:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161290 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.841024:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161290 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.841360:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.841377:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161204 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef28a expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.841386:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.841421:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161204 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef28a expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.841435:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161204 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef28a expref: 70 pid: 9040 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.841450:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382330240:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572538.841563:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.844840:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382330240:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572538.844875:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.847682:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967520 gen 2 00000004:00080000:1.0:1788572538.847688:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967520 committed transno 4294967520 00000100:00100000:1.0:1788572538.847711:0:22536:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627dfdbc380 x1875453382328704/t4294967525(4294967525) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1788572554 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572538.847722:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2399f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382330368:0@lo:1000:mdt00_004.0 00000001:00080000:1.0:1788572538.848083:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970110 00000100:00100000:1.0:1788572538.848123:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382330496:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572538.848147:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572538.848167:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194480 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.848855:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382330496:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572538.848885:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.848894:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.848903:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.848909:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.848913:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.848919:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.848931:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627d599a400) 00010000:00010000:1.0:1788572538.848932:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.848979:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.848984:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb161297 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849069:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849075:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849079:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849081:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.849083:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849086:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849092:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7f:0x0].0x0 (ffff9627d599a700) 00010000:00010000:1.0:1788572538.849093:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.849096:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.849099:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935800/0x85ace67feb15ea18 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572538.849837:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967526 is committed 00000001:00080000:3.0:1788572538.849857:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967527 is committed 00000100:00100000:1.0:1788572538.853438:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.853504:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360239232 00000100:00100000:3.0:1788572538.853510:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360239232 00000100:00100000:3.0:1788572538.853560:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3722 00000100:00100000:3.0:1788572538.853569:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35882:x1875453360239232:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.853607:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.853630:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1611da lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef275 expref: 76 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.853653:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572538.853658:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572538.853683:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.853693:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.853701:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572538.853811:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007fd69d88 reply state 00000000dcb1b135 transno 4294970114 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.853819:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572538.853921:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35882:x1875453360239232:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 352us (484us total) trans 4294970114 rc 0/0 00000100:00100000:3.0:1788572538.853928:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3722 00010000:00010000:3.0:1788572538.854638:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572538.854645:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1612a5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.854653:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572538.854657:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1612a5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.854666:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1612a5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.854678:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:3.0:1788572538.854682:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572538.854687:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1612a5 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.854693:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1612a5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.854711:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.854712:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.854714:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.856827:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.856841:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.856854:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.856858:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.856862:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.856868:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.856881:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572538.856883:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.856887:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 2/0,0 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.856891:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1612ac lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.858639:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360239360 00000100:00100000:1.0:1788572538.858650:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360239360 00000100:00100000:1.0:1788572538.858748:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 933 00000100:00100000:1.0:1788572538.858759:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360239360:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572538.859032:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:1.0:1788572538.859036:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb mtime from disk: 1788572538 -> 1788572520 00002000:00010000:1.0:1788572538.859039:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:1.0:1788572538.859042:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb ctime from disk: 1788572538 -> 1788572524 00002000:00010000:1.0:1788572538.859044:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x3d:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:1.0:1788572538.859113:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360239360:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 355us (1290us total) trans 4294967530 rc 0/0 00000100:00100000:1.0:1788572538.859121:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 933 00000100:00100000:1.0:1788572538.861686:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.864349:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360239744 00000100:00100000:3.0:1788572538.864357:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360239744 00010000:00010000:3.0:1788572538.864391:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef228 expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.864447:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 887 00000100:00100000:3.0:1788572538.864454:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2350:x1875453360239744:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.864464:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.864481:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef228 expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.864490:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef228 expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.864510:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1610e5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef228 expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.864525:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572538.864529:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1612b3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.864538:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1612b3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.864546:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1612b3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.864575:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.864615:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2350:x1875453360239744:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 161us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.864620:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 887 00000100:00100000:1.0:1788572538.866444:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360239872 00000100:00100000:1.0:1788572538.866451:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360239872 00000100:00100000:1.0:1788572538.866535:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 934 00000100:00100000:1.0:1788572538.866547:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360239872:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.866592:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/1,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.866608:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 12 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.866614:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572538.866617:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572538.866620:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 4/1,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.866647:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 12 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.866658:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005e322adb/0x85ace67feb16120b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef298 expref: 12 pid: 6679 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.866673:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382330752:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572538.866732:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194480 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.871661:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.871793:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382330752:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572538.871819:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/1,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872616:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872627:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872655:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872664:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872673:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872679:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572538.872683:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872696:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872709:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.872716:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad478b03/0x85ace67feb1612ba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.872784:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360239872:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 6237us (6405us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.872793:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 934 00000100:00100000:1.0:1788572538.879924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572538.885687:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.885714:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.885726:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.885999:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.886048:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 75 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.886056:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.886094:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 75 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.886107:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 75 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.886130:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382330880:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572538.886217:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.889393:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382330880:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572538.889421:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.889455:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360240384 00000100:00100000:3.0:1788572538.889462:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360240384 00010000:00010000:3.0:1788572538.889514:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.889592:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 889 00000100:00100000:3.0:1788572538.889602:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2350:x1875453360240384:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572538.889627:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572538.889635:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.889647:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.889673:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161147 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef24b expref: 76 pid: 22536 timeout: 986 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.889689:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572538.889695:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.889704:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.889711:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.889739:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572538.889798:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2350:x1875453360240384:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 197us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.889806:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 889 00010000:00010000:1.0:1788572538.890171:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.890179:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572538.890405:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970115 00000001:00080000:1.0:1788572538.890665:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970116 00000001:00080000:1.0:1788572538.890713:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970117 00000100:00100000:1.0:1788572538.890799:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382331008:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572538.890846:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572538.890921:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194480 going to sleep for 11 seconds 00000100:00100000:3.0:1788572538.891433:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967527 gen 2 00000004:00080000:3.0:1788572538.891439:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfdbc380, transno 4294967525 00000004:00080000:3.0:1788572538.891454:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfdbf100, transno 4294967526 00000004:00080000:3.0:1788572538.891458:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfdbdc00, transno 4294967527 00000100:00100000:3.0:1788572538.891465:0:5837:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627dfdbed80 x1875453382330496/t4294967529(4294967529) o1000->lustre-OST0001-osc-MDT0000@0@lo:24/4 lens 408/4320 e 0 to 0 dl 1788572554 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mdt00_004.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572538.891485:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382331008:0@lo:1000:mdt00_000.0 00000004:00080000:1.0:1788572538.891754:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 3 done 00000001:00080000:3.0:1788572538.893994:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970118 00000100:00100000:3.0:1788572538.894116:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382331136:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572538.894182:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572538.894418:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e77346c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572538.894889:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382331136:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1788572538.894931:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.894951:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.894960:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.894967:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.894972:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.894980:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.894998:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572538.895001:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.895023:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895030:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1612c8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895060:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895067:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895072:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895076:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.895079:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895084:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895096:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572538.895098:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572538.895101:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895107:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935e00/0x85ace67feb15e9a1 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895127:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 3/0,1 mode: --/EX res: [0x200000401:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895139:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895148:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895155:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895161:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895167:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895171:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.895173:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895179:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895187:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572538.895190:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.895193:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 2/0,0 mode: --/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895198:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612cf lrc: 0/0,0 mode: --/EX res: [0x200000401:0x80:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895207:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895212:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895217:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895223:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.895226:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895231:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895239:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572538.895241:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572538.895255:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.895261:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612c1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.895540:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35882:x1875453360240256:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 11134us (15623us total) trans 4294970119 rc 0/0 00000100:00100000:1.0:1788572538.895549:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3724 00000020:00080000:1.0:1788572538.895553:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572538.898989:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360240640 00000100:00100000:3.0:1788572538.898998:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360240640 00000100:00100000:3.0:1788572538.899069:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 841 00000100:00100000:3.0:1788572538.899079:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35882:x1875453360240640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.903292:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572538.903304:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1612d6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.903315:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572538.903319:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1612d6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.903330:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1612d6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.903347:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6c:0x0].0x0 (ffff9627e7a95400) 00010000:00010000:3.0:1788572538.903350:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572538.903355:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1612d6 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.903361:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1612d6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.905702:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360240768 00000100:00100000:1.0:1788572538.905713:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360240768 00000100:00100000:1.0:1788572538.905767:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3725 00000100:00100000:1.0:1788572538.905777:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35887:x1875453360240768:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572538.905811:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x78:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572538.905920:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35887:x1875453360240768:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 144us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.905927:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3725 00000100:00100000:1.0:1788572538.906649:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3727 00000100:00100000:1.0:1788572538.906657:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35885:x1875453360241024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572538.906671:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x79:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572538.914363:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35885:x1875453360241024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 7692us (7903us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.914380:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3727 00000100:00100000:1.0:1788572538.919597:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360241152 00000100:00100000:1.0:1788572538.919609:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360241152 00000100:00100000:1.0:1788572538.919664:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360241280 00000100:00100000:1.0:1788572538.919667:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360241280 00000100:00100000:1.0:1788572538.919747:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3728 00000100:00100000:1.0:1788572538.919759:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35887:x1875453360241152:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.919864:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919883:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919894:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919908:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919915:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919922:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919951:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x78:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919958:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.919965:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.920041:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x78:0x0]: returning size 3 00000004:00200000:1.0:1788572538.920046:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572538.920059:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920071:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920080:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920086:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920092:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.920096:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920103:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920114:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572538.920116:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.920121:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920127:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1612dd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920137:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920143:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920148:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920153:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.920157:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920162:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920168:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x714b7b7f (ffff9627d0ee2e00) 00010000:00010000:1.0:1788572538.920170:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.920173:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920178:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1612e4 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x714b7b7f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920202:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920208:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920213:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920218:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.920222:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920227:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920235:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x78:0x0].0x0 (ffff9627fb1e7f00) 00010000:00010000:1.0:1788572538.920237:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.920240:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920246:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1612eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x78:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.920314:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35887:x1875453360241152:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 557us (1200us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.920322:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3728 00000100:00100000:1.0:1788572538.920340:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3729 00000100:00100000:1.0:1788572538.920347:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35886:x1875453360241280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.920373:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920382:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920389:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920400:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920406:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920412:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920427:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920439:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920445:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.920463:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7a:0x0]: returning size 3 00000004:00200000:1.0:1788572538.920467:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572538.920483:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920490:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920495:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920501:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920506:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.920508:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920514:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920522:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572538.920525:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.920527:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920536:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1612f2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920545:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920550:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920556:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920562:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.920565:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920570:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920577:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x4432fb55 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572538.920579:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.920581:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920587:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1612f9 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4432fb55 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920598:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920604:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920610:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920615:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.920617:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920623:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920629:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627d599a800) 00010000:00010000:1.0:1788572538.920631:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.920633:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.920639:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161300 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572538.920670:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35886:x1875453360241280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 323us (1342us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.920675:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3729 00000100:00100000:1.0:1788572538.924968:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.935506:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.935547:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.943138:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.943714:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.950192:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.954203:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.955256:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360242944 00000100:00100000:1.0:1788572538.955266:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.955270:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360242944 00000100:00100000:1.0:1788572538.955334:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360243072 00000100:00100000:1.0:1788572538.955339:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360243072 00000100:00100000:1.0:1788572538.955387:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3738 00000100:00100000:3.0:1788572538.955396:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3739 00000100:00100000:1.0:1788572538.955403:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35886:x1875453360243072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:3.0:1788572538.955411:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35887:x1875453360242944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.955427:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.955428:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.955433:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572538.955453:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161377 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2ec expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.955460:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161370 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2de expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.955492:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572538.955498:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572538.955508:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16137e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2e5 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955543:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955559:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955571:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955586:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955593:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955599:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955959:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161393 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955969:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161393 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.955977:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161393 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572538.956000:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:1.0:1788572538.956037:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572538.956056:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161393 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956067:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956074:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956080:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956086:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.956091:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956099:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956109:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572538.956112:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.956116:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956123:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161385 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956132:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956143:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956149:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956154:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572538.956157:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956164:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956174:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572538.956177:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572538.956180:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956188:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16138c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956218:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161377 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ec expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956230:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161393 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2ec expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.956237:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572538.956239:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572538.956242:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e000, rc 0) 00000100:00100000:1.0:1788572538.956304:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35886:x1875453360243072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 907us (1042us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572538.956313:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3738 00000004:00080000:3.0:1788572538.956404:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x3f:0x0], index 0: 0 00010000:00010000:3.0:1788572538.956819:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16139a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.956831:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16139a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.956840:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16139a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.956895:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16137e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2e5 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.956906:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16139a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2e5 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.956912:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6f:0x0].0x0 (ffff9627d599a900) 00010000:00010000:3.0:1788572538.956914:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572538.956916:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:3.0:1788572538.956944:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35887:x1875453360242944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1539us (2757us total) trans 4294970125 rc 0/0 00000100:00100000:3.0:1788572538.956952:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3739 00000100:00100000:1.0:1788572538.965813:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.965856:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.966075:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3741 00000100:00100000:3.0:1788572538.966085:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35885:x1875453360243456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.966096:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572538.966116:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1613a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef2f3 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966172:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966184:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966204:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966220:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966229:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966237:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966611:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1613cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966624:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1613cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966642:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1613cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572538.966670:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:3.0:1788572538.966675:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572538.966694:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1613cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966710:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966723:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966728:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966731:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.966733:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966737:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966743:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572538.966745:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.966747:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966751:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1613af lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966761:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966766:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966770:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966773:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572538.966774:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966778:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966781:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d599a100) 00010000:00010000:3.0:1788572538.966783:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572538.966785:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966788:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1613b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966807:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1613a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2f3 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966815:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1613cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2f3 expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.966834:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572538.966840:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572538.966845:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:3.0:1788572538.966873:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35885:x1875453360243456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 790us (1016us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572538.966880:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3741 00000100:00100000:1.0:1788572538.968797:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.969550:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360243584 00000100:00100000:3.0:1788572538.969559:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360243584 00000100:00100000:1.0:1788572538.969732:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.974140:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.978683:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.978947:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360244096 00000100:00100000:3.0:1788572538.978959:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360244096 00000100:00100000:3.0:1788572538.979089:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3744 00000100:00100000:3.0:1788572538.979100:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35887:x1875453360244096:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572538.979196:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1613f5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.979214:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1613f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572538.979226:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1613f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572538.979664:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].51, 88 off12200 00000004:00010000:3.0:1788572538.979761:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006fc4e72e reply state 0000000004925e28 transno 4294970127 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1613f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.979771:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572538.979903:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35887:x1875453360244096:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 802us (5772us total) trans 4294970127 rc 0/0 00000100:00100000:3.0:1788572538.979915:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3744 00000100:00100000:3.0:1788572538.982210:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360244480 00000100:00100000:3.0:1788572538.982234:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360244480 00000100:00100000:1.0:1788572538.982536:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360244224 00000100:00100000:1.0:1788572538.982541:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360244224 00000100:00100000:1.0:1788572538.982582:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3745 00000100:00100000:1.0:1788572538.982593:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35886:x1875453360244480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572538.982617:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572538.982636:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1613fc lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef30f expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572538.984638:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x40:0x0], index 0: 0 00010000:00010000:1.0:1788572538.984860:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161418 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.984872:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161418 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.984880:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161418 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572538.986333:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572538.988932:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1613fc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef30f expref: 94 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.989347:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161418 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef30f expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572538.989356:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572538.989359:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572538.989362:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:1.0:1788572538.989418:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35886:x1875453360244480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6827us (10743us total) trans 4294970128 rc 0/0 00000100:00100000:1.0:1788572538.989426:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3745 00000100:00100000:3.0:1788572538.991733:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572538.992436:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572538.992514:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360244736 00000100:00100000:3.0:1788572538.992519:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360244736 00000100:00100000:3.0:1788572538.992554:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 935 00000100:00100000:3.0:1788572538.992562:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360244736:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572538.992869:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360244736:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 306us (432us total) trans 4294967531 rc 0/0 00000100:00100000:3.0:1788572538.992877:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 935 00000100:00100000:1.0:1788572538.995439:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.002649:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.002690:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.002723:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.002751:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360245504 00000100:00100000:1.0:1788572539.002761:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360245504 00000100:00100000:1.0:1788572539.002816:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360245632 00000100:00100000:1.0:1788572539.002820:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360245632 00000100:00100000:1.0:1788572539.002838:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360245888 00000100:00100000:1.0:1788572539.002841:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360245888 00000100:00100000:1.0:1788572539.002860:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3749 00000100:00100000:1.0:1788572539.002870:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35887:x1875453360245504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.002986:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35887:x1875453360245504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 116us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.002994:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3749 00000100:00100000:1.0:1788572539.003030:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3750 00000100:00100000:1.0:1788572539.003038:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35886:x1875453360245632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.003051:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.003072:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161442 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef31d expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.003107:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.003121:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.003132:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.003223:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161442 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 94 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.003256:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.003265:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:1.0:1788572539.003268:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572539.003272:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc 0) 00000100:00100000:1.0:1788572539.003478:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35886:x1875453360245632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 431us (777us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.003485:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3750 00000100:00080000:1.0:1788572539.003503:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788572539.004127:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572539.007422:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.007460:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.007560:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360246272 00000100:00100000:3.0:1788572539.007564:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360246272 00000100:00100000:3.0:1788572539.007608:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3752 00000100:00100000:3.0:1788572539.007618:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35886:x1875453360246272:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.007698:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.007712:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.007724:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572539.008217:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].52, 88 off12288 00000004:00010000:3.0:1788572539.008318:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000075c832a5 reply state 00000000306f0819 transno 4294970130 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.008327:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572539.008373:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35886:x1875453360246272:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 756us (911us total) trans 4294970130 rc 0/0 00000100:00100000:3.0:1788572539.008379:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3752 00010000:00010000:3.0:1788572539.009885:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.009898:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.009905:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.009911:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.009915:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.009921:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.009936:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:3.0:1788572539.009938:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1788572539.010314:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.010325:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161465 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.012766:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.012848:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.018227:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.018285:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.018327:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360246400 00000100:00100000:3.0:1788572539.018338:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360246400 00000100:00100000:3.0:1788572539.018399:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360246912 00000100:00100000:3.0:1788572539.018403:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360246912 00000100:00100000:3.0:1788572539.018811:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3756 00000100:00100000:3.0:1788572539.018820:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35887:x1875453360246912:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.021078:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35887:x1875453360246912:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 2256us (2800us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.021087:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3756 00000100:00100000:1.0:1788572539.035719:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.035749:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.035811:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.035908:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 938 00000100:00100000:1.0:1788572539.035915:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360247040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.036398:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360247040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 483us (646us total) trans 4294967532 rc 0/0 00000100:00100000:1.0:1788572539.036407:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 938 00000100:00100000:3.0:1788572539.042586:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.042743:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.042767:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360247808 00000100:00100000:3.0:1788572539.042776:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360247808 00010000:00010000:3.0:1788572539.042815:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb161473 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef332 expref: 15 pid: 6682 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.042854:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 890 00000100:00100000:3.0:1788572539.042864:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360247808:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.042877:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.042900:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb161473 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef332 expref: 15 pid: 6682 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.042915:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb161473 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef332 expref: 15 pid: 6682 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.042942:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.042946:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.043050:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 15 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.043063:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 15 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.043104:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 15 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.043119:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382331520:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572539.043191:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572539.054319:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.054361:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.054813:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382331520:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572539.054839:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.054901:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360247808:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 12037us (12157us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.054913:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 890 00000100:00100000:3.0:1788572539.056889:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.061452:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360248448 00000100:00100000:1.0:1788572539.061463:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360248448 00000100:00100000:1.0:1788572539.061536:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3760 00000100:00100000:1.0:1788572539.061546:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35886:x1875453360248448:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.061569:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.061592:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1614a4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef34e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.061665:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1614ab lrc: 3/1,0 mode: --/PR res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.061680:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1614ab lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.061692:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1614ab lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572539.061728:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572539.061747:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1614ab lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.061775:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1614a4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef34e expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.061792:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1614ab lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef34e expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.061801:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:1.0:1788572539.061804:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572539.061808:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:1.0:1788572539.061872:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35886:x1875453360248448:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 327us (553us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.061879:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3760 00000100:00100000:3.0:1788572539.063663:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.067449:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360248960 00000100:00100000:1.0:1788572539.067460:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360248960 00000100:00100000:1.0:1788572539.067535:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 844 00000100:00100000:1.0:1788572539.067545:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35885:x1875453360248960:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.068001:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35885:x1875453360248960:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 456us (769us total) trans 4294970133 rc 0/0 00000100:00100000:1.0:1788572539.068028:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 844 00000100:00100000:1.0:1788572539.070928:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.070967:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360249088 00000100:00100000:1.0:1788572539.070976:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360249088 00000100:00100000:1.0:1788572539.071042:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 940 00000100:00100000:1.0:1788572539.071052:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360249088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.071069:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.071089:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb1614b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef355 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572539.071116:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x40:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737a, blocks: 0x1 00010000:00010000:1.0:1788572539.071133:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=887s v=(0 0) 00010000:00010000:1.0:1788572539.071138:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.071146:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb1614b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef355 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.071165:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb1614b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef355 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.071178:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:1.0:1788572539.071225:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360249088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 175us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.071232:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 940 00000100:00100000:1.0:1788572539.081070:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.081109:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.081140:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360249856 00000100:00100000:1.0:1788572539.081151:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360249856 00000100:00100000:1.0:1788572539.081207:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 941 00000100:00100000:1.0:1788572539.081216:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35886:x1875453360249856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.081250:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.081273:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000142632fa/0x85ace67feb1614b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef363 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.081306:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb1614b2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef355 expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.081316:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.081321:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.081327:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000142632fa/0x85ace67feb1614b9 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef363 expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.081363:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb1614b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef355 expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.081377:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb1614b2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef355 expref: 17 pid: 20911 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.081392:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382331648:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572539.081466:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.086161:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382331648:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.086195:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000142632fa/0x85ace67feb1614b9 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef363 expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.086216:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:3.0:1788572539.087583:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360249728 00000100:00100000:3.0:1788572539.087590:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360249728 00000100:00100000:3.0:1788572539.087634:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3764 00000100:00100000:3.0:1788572539.087641:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35885:x1875453360249728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.087659:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.087676:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1614c0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x70:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef35c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.087739:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35886:x1875453360249856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6523us (6628us total) trans 0 rc 0/0 00010000:00010000:3.0:1788572539.087740:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1614c7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x70:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.087746:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 941 00010000:00010000:3.0:1788572539.087752:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1614c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x70:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.087762:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1614c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x70:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.087804:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.087824:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1614c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x70:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.087848:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1614c0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef35c expref: 103 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.087862:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1614c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x70:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef35c expref: 102 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.087869:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x70:0x0].0x0 (ffff9627e7a95600) 00010000:00010000:3.0:1788572539.087871:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572539.087874:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:3.0:1788572539.087896:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35885:x1875453360249728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 256us (6837us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.087902:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3764 00000100:00100000:1.0:1788572539.093655:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.094987:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.095080:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360250240 00000100:00100000:3.0:1788572539.095087:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360250240 00002000:00100000:3.0:1788572539.095123:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x40:0x0] (0->2) req@ffff9626c1bce300 x1875453360250240/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/0 e 0 to 0 dl 1788572550 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572539.095134:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bce300 with x1875453360250240 ext(0->2) 00002000:00010000:3.0:1788572539.095139:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1bce300 00000100:00100000:3.0:1788572539.095167:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 221 00000100:00100000:3.0:1788572539.095175:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360250240:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.095740:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360250240:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 564us (754us total) trans 4294967534 rc 0/0 00000100:00100000:3.0:1788572539.095748:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 221 00002000:00100000:3.0:1788572539.095755:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x40:0x0] (0->2) req@ffff9626c1bce300 x1875453360250240/t4294967534(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/448 e 0 to 0 dl 1788572550 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572539.095764:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bce300 with x1875453360250240 ext(0->2) 00002000:00010000:3.0:1788572539.095767:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1bce300 00000100:00100000:3.0:1788572539.098373:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.100844:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360250624 00000100:00100000:1.0:1788572539.100855:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360250624 00000100:00100000:1.0:1788572539.100916:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3766 00000100:00100000:1.0:1788572539.100928:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35886:x1875453360250624:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.101082:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35886:x1875453360250624:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 156us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.101090:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3766 00000100:00100000:1.0:1788572539.103543:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.109682:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.120322:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.120353:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360251008 00000100:00100000:3.0:1788572539.120362:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360251008 00010000:00010000:3.0:1788572539.120403:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1614ce lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef36a expref: 15 pid: 6679 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.120440:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 892 00000100:00100000:3.0:1788572539.120450:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360251008:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.120464:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.120499:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1614ce lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef36a expref: 15 pid: 6679 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.120514:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1614ce lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef36a expref: 15 pid: 6679 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.120542:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.120547:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.120557:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.120567:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.120604:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.120621:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382332032:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572539.120701:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e77346c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572539.121032:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1614ce lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef36a expref: 15 pid: 6679 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.121050:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7559880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382331904:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.121076:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.121086:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bac400, rc 0) 00000100:00100000:1.0:1788572539.121121:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35885:x1875453360250752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 17415us (17588us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.121129:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 943 00000100:00100000:3.0:1788572539.132607:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382332032:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572539.132634:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.132696:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360251008:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 12247us (12381us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.132708:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 892 00000100:00100000:1.0:1788572539.134185:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360251136 00000100:00100000:1.0:1788572539.134192:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360251136 00002000:00100000:1.0:1788572539.134241:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3f:0x0] (0->2) req@ffff962801f97480 x1875453360251136/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/0 e 0 to 0 dl 1788572550 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572539.134252:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f97480 with x1875453360251136 ext(0->2) 00002000:00010000:1.0:1788572539.134258:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f97480 00000100:00100000:1.0:1788572539.134291:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 222 00000100:00100000:1.0:1788572539.134302:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360251136:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.134744:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360251136:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 442us (998us total) trans 4294967533 rc 0/0 00000100:00100000:1.0:1788572539.134752:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 222 00002000:00100000:1.0:1788572539.134759:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3f:0x0] (0->2) req@ffff962801f97480 x1875453360251136/t4294967533(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/448 e 0 to 0 dl 1788572550 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572539.134769:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f97480 with x1875453360251136 ext(0->2) 00002000:00010000:1.0:1788572539.134772:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f97480 00000100:00100000:1.0:1788572539.141379:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.143580:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360251264 00000100:00100000:3.0:1788572539.143595:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360251264 00000100:00100000:3.0:1788572539.143652:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3767 00000100:00100000:3.0:1788572539.143664:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35885:x1875453360251264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.143801:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35885:x1875453360251264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 140us (2431us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.143809:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3767 00000100:00100000:1.0:1788572539.152366:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.159489:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35887:x1875453360248704:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 95371us (95812us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.159506:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3761 00000001:00080000:1.0:1788572539.159548:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970131 is committed 00000020:00080000:1.0:1788572539.159569:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 87 batchid updated 00000020:00080000:1.0:1788572539.159602:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d1e51cc0 refcount 2 committed 0 result 0 batchid 89 00000020:00080000:1.0:1788572539.159606:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31deb00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d1e51000 00000020:00080000:1.0:1788572539.159610:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627dfc60400 00000001:00080000:1.0:1788572539.159620:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970132 is committed 00000020:00080000:1.0:1788572539.159650:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2eb5840 refcount 2 committed 0 result 0 batchid 88 00000020:00080000:1.0:1788572539.159653:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627fb54b900 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2eb5000 00000020:00080000:1.0:1788572539.159657:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627fb54b600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f5ed9000 00000020:00080000:1.0:1788572539.159676:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627e7734e40 refcount 2 committed 0 result 0 batchid 90 00000020:00080000:1.0:1788572539.159679:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8adc00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627e77343c0 00000020:00080000:1.0:1788572539.159682:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9628020e2800 00000100:00100000:3.0:1788572539.165886:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360250880:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 55745us (56203us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.165904:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 944 00000100:00100000:1.0:1788572539.174648:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360251520 00000100:00100000:1.0:1788572539.174666:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360251520 00000100:00100000:3.0:1788572539.178130:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572539.178173:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.178186:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.178193:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.178199:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.178204:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.178214:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.178230:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:3.0:1788572539.178234:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.178238:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 2/0,0 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.178245:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1614dc lrc: 0/0,0 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.184084:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360252160 00000100:00100000:1.0:1788572539.184094:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360252160 00000100:00100000:1.0:1788572539.184145:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3769 00000100:00100000:1.0:1788572539.184155:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35887:x1875453360252160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.184218:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.184255:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb16146c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef324 expref: 66 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.184385:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6f:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:1.0:1788572539.184391:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572539.184422:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1614e3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.184437:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1614e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.184447:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1614e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572539.184671:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c364c1b0 reply state 0000000022f2776e transno 4294970135 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1614e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.184681:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572539.184781:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35887:x1875453360252160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 626us (928us total) trans 4294970135 rc 0/0 00000100:00100000:1.0:1788572539.184789:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3769 00000100:00100000:1.0:1788572539.187656:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.187734:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360252416 00000100:00100000:3.0:1788572539.187746:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360252416 00000100:00100000:3.0:1788572539.187806:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3770 00000100:00100000:3.0:1788572539.187816:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35886:x1875453360252416:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.187903:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1614ea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.187920:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.187929:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1614ea lrc: 4/0,1 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.187965:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.187979:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 67 pid: 22536 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.187994:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff7480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382332160:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.188102:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c63cc780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572539.196916:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.197168:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360252544 00000100:00100000:1.0:1788572539.197176:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360252544 00010000:00010000:1.0:1788572539.197213:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 68 pid: 22536 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.197329:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 893 00000100:00100000:1.0:1788572539.197339:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360252544:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.197354:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.197360:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 68 pid: 22536 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.197372:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 68 pid: 22536 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.197389:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161449 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x71:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef31d expref: 68 pid: 22536 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.197408:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:1.0:1788572539.197412:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1614ea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.197422:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1614ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.197428:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1614ea lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.197450:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.197685:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360252544:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 346us (775us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.197693:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 893 00000100:00100000:3.0:1788572539.201867:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360252672 00000100:00100000:3.0:1788572539.201877:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360252672 00000100:00100000:3.0:1788572539.201924:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 948 00000100:00100000:3.0:1788572539.201934:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360252672:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572539.202169:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3f:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572539.202173:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3f:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:3.0:1788572539.202176:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x3f:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:3.0:1788572539.202177:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3f:0x0] updating lvb ctime from disk: 1788572538 -> 1788572524 00002000:00010000:3.0:1788572539.202180:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x3f:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572539.202228:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360252672:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 295us (466us total) trans 4294967536 rc 0/0 00000100:00100000:3.0:1788572539.202235:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 948 00000100:00100000:3.0:1788572539.208360:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000001:00080000:3.0:1788572539.212874:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967531 is committed 00000001:00080000:3.0:1788572539.212891:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967533 is committed 00000100:00100000:3.0:1788572539.219664:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.219730:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.219807:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360253184 00000100:00100000:1.0:1788572539.219817:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360253184 00010000:00010000:1.0:1788572539.219856:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1613d9 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2fa expref: 66 pid: 5838 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.219898:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 894 00000100:00100000:1.0:1788572539.219908:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360253184:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.219922:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.219929:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1613d9 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2fa expref: 66 pid: 5838 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.219953:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1613d9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef2fa expref: 66 pid: 5838 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.219978:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6f:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:1.0:1788572539.219983:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1614f1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.219993:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1614f1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.220001:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1614f1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.220048:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.220095:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360253184:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 190us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.220102:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 894 00000100:00100000:1.0:1788572539.224670:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.224708:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.224807:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360253568 00000100:00100000:3.0:1788572539.224819:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360253568 00010000:00010000:3.0:1788572539.224866:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000142632fa/0x85ace67feb1614b9 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef363 expref: 14 pid: 10416 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.224918:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 895 00000100:00100000:3.0:1788572539.224929:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360253568:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.224954:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.224978:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000142632fa/0x85ace67feb1614b9 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef363 expref: 14 pid: 10416 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.224994:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000142632fa/0x85ace67feb1614b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef363 expref: 14 pid: 10416 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.225048:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572539.225053:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572539.225059:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb1614f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.225074:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb1614f8 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.225097:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.225151:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360253568:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 224us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.225159:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 895 00000100:00100000:3.0:1788572539.231659:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572539.240924:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.240951:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.240960:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.240966:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.240972:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.240979:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.240997:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x70:0x0].0x0 (ffff9627e7a95600) 00010000:00010000:3.0:1788572539.241000:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.241039:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 2/0,0 mode: --/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.241051:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1614ff lrc: 0/0,0 mode: --/PW res: [0x200000402:0x70:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.242101:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.242295:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360253824 00000100:00100000:1.0:1788572539.242307:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360253824 00000100:00100000:1.0:1788572539.242363:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 951 00000100:00100000:1.0:1788572539.242373:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360253824:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572539.242645:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x3f:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572539.242649:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x3f:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:1.0:1788572539.242651:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x3f:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:1.0:1788572539.242653:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x3f:0x0] updating lvb ctime from disk: 1788572538 -> 1788572524 00002000:00010000:1.0:1788572539.242656:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x3f:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572539.242729:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360253824:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 357us (634us total) trans 4294967534 rc 0/0 00000100:00100000:1.0:1788572539.242740:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 951 00000100:00100000:3.0:1788572539.247425:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.247835:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360253952 00000100:00100000:1.0:1788572539.247855:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360253952 00010000:00010000:1.0:1788572539.247899:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 13 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.247952:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 896 00000100:00100000:1.0:1788572539.247963:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360253952:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.247979:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.248001:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 13 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.248142:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 13 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.248166:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097f34723/0x85ace67feb16148f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef340 expref: 13 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.248183:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.248188:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.248194:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb161506 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.248205:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb161506 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.248342:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.248403:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360253952:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 441us (984us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.248411:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 896 00000100:00100000:3.0:1788572539.255795:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.255894:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360254208 00000100:00100000:1.0:1788572539.255906:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360254208 00000100:00100000:1.0:1788572539.255971:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3773 00000100:00100000:1.0:1788572539.255982:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35886:x1875453360254208:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.256088:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256105:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256117:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256358:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256370:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256377:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256409:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.256428:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 65 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.256441:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382332672:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572539.256530:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff5483c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.259110:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.259252:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360254464 00000100:00100000:3.0:1788572539.259263:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360254464 00000100:00100000:3.0:1788572539.259320:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3774 00000100:00100000:3.0:1788572539.259330:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35887:x1875453360254464:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.259416:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb16151b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x78:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259434:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb16151b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x78:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259445:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb16151b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x78:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259678:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259691:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259699:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259734:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.259745:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 67 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.259759:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382332800:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.259842:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734780 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.265391:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382332672:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.265413:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.266059:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360254976 00000100:00100000:3.0:1788572539.266067:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360254976 00010000:00010000:3.0:1788572539.266107:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 69 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.266145:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360254848 00000100:00100000:3.0:1788572539.266149:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360254848 00010000:00010000:3.0:1788572539.266163:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 70 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.266192:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 897 00000100:00100000:3.0:1788572539.266201:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360254976:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.266216:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.266235:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 70 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266246:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 70 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266268:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6f:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572539.266273:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266283:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266290:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266308:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.266363:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360254976:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 162us (1944us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.266371:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 897 00000100:00100000:3.0:1788572539.266389:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 898 00000100:00100000:3.0:1788572539.266396:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0facc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2350:x1875453360254848:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.266406:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.266410:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 69 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266420:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 69 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266432:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16147a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef339 expref: 69 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266447:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:3.0:1788572539.266452:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266459:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266466:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266501:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.266520:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0facc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360254848:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 125us (2076us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.266528:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 898 00010000:00010000:3.0:1788572539.266598:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb16142d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef316 expref: 67 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.266608:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382332800:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.266621:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266629:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.266634:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161522 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572539.266832:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970138 00000001:00080000:3.0:1788572539.267536:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970139 00000001:00080000:3.0:1788572539.267588:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970140 00000100:00100000:3.0:1788572539.267671:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382333056:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572539.267720:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.267742:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a540 going to sleep for 11 seconds 00010000:00010000:3.0:1788572539.267764:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.267776:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572539.267858:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970137 00000001:00080000:3.0:1788572539.269619:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970142 00000001:00080000:3.0:1788572539.269690:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970143 00000100:00100000:3.0:1788572539.269764:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382333184:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572539.269801:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.269933:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d2241e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.274765:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.275787:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382332928:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.275819:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161529 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x70:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.276693:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382333184 00000100:00100000:1.0:1788572539.276700:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382333184 00000100:00100000:1.0:1788572539.276738:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382333312 00000100:00100000:1.0:1788572539.276741:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382333312 00000100:00100000:1.0:1788572539.276756:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:1.0:1788572539.276762:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382333184:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572539.278489:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382333184:12345-0@lo:1000:mdt00_003.0 Request processed in 1723us (8683us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.278500:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1788572539.282738:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360255360 00000100:00100000:2.0:1788572539.282750:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360255360 00000100:00100000:2.0:1788572539.282812:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 845 00000100:00100000:2.0:1788572539.282823:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35887:x1875453360255360:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572539.283744:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x6f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572539.283875:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970146 00010000:00010000:2.0:1788572539.283924:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161537 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284337:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161537 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284352:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161537 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284362:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161537 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284369:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161537 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284375:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161537 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284429:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572539.284547:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35887:x1875453360255360:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1723us (2501us total) trans 4294970147 rc 0/0 00000100:00100000:2.0:1788572539.284557:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 845 00000020:00080000:2.0:1788572539.284562:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788572539.284595:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360255488 00000100:00100000:2.0:1788572539.284601:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360255488 00000100:00100000:2.0:1788572539.284634:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 952 00000100:00100000:2.0:1788572539.284642:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360255488:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.284670:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/1,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284688:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 11 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284697:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572539.284701:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572539.284707:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 4/1,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284740:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 11 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.284755:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 11 pid: 20911 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.284769:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382333440:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572539.284818:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8180 going to sleep for 11 seconds 00000100:00100000:3.0:1788572539.285270:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382333184:0@lo:1000:mdt00_003.0 00000001:00080000:3.0:1788572539.285644:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970144 00000100:00100000:3.0:1788572539.285711:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382333568:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572539.285765:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.285796:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572539.290830:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382333440:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.290875:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/1,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.291656:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.291724:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360255616 00000100:00100000:3.0:1788572539.291731:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360255616 00010000:00010000:3.0:1788572539.291769:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 12 pid: 20911 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.291818:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 900 00000100:00100000:3.0:1788572539.291828:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360255616:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.291842:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.291878:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 12 pid: 20911 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.291892:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 12 pid: 20911 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.291917:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000049f900c8/0x85ace67feb1614d5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef371 expref: 12 pid: 20911 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.291934:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572539.291938:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572539.291950:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.291961:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.291987:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.292051:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360255616:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 223us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.292059:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 900 00010000:00010000:3.0:1788572539.292082:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292091:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292115:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292123:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292132:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292139:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572539.292143:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292152:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292164:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.292172:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb16153e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.292407:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382333568 00000100:00100000:2.0:1788572539.292413:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382333568 00000100:00100000:2.0:1788572539.292447:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:2.0:1788572539.292455:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382333568:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572539.292702:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:9040:x1875453382333568:12345-0@lo:1000:mdt00_003.0 Request processed in 247us (6936us total) trans 4294967538 rc 0/0 00000100:00100000:2.0:1788572539.292710:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:3.0:1788572539.292772:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360255488:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 8129us (10633us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.292781:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 952 00000100:00100000:3.0:1788572539.292832:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382333568:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1788572539.293619:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293631:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293639:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293646:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.293650:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293659:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293672:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d0ee2e00) 00010000:00010000:3.0:1788572539.293675:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572539.293680:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293686:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161514 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x71:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293720:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293726:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293732:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293737:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.293740:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293746:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293755:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572539.293758:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572539.293761:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293766:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935c00/0x85ace67feb15eab9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293789:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293800:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293808:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293815:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293821:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293827:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293833:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.293836:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293842:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293865:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572539.293867:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.293870:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293876:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161545 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293885:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293891:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293897:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293902:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.293906:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293912:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293919:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572539.293922:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572539.293924:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.293931:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16150d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.294101:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35886:x1875453360254208:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 38119us (38313us total) trans 4294970148 rc 0/0 00000100:00100000:3.0:1788572539.294110:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3773 00000020:00080000:3.0:1788572539.294115:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572539.297409:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.297453:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.297777:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360256128 00000100:00100000:3.0:1788572539.297786:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360256128 00000100:00100000:3.0:1788572539.297840:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 846 00000100:00100000:3.0:1788572539.297859:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e65c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35886:x1875453360256128:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572539.298215:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x71:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572539.298263:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970149 00010000:00010000:3.0:1788572539.298296:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298309:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298319:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298328:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298334:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298340:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298355:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572539.298429:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e65c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:35886:x1875453360256128:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 580us (975us total) trans 4294970150 rc 0/0 00000100:00100000:3.0:1788572539.298436:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 846 00000020:00080000:3.0:1788572539.298440:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572539.298497:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572539.298504:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298510:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572539.298513:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298520:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298529:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x71:0x0].0x0 (ffff9627d599a300) 00010000:00010000:3.0:1788572539.298531:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572539.298535:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298544:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb16154c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x71:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.298557:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572539.298558:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572539.298560:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572539.299175:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360255872 00000100:00100000:2.0:1788572539.299182:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360255872 00000100:00100000:2.0:1788572539.299226:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3776 00000100:00100000:2.0:1788572539.299235:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:35885:x1875453360255872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.299289:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299301:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299310:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299444:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x70:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299457:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 76 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299465:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 4/0,1 mode: --/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299515:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 76 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.299529:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 76 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.299550:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382333696:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572539.299621:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b83c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.304297:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382333696:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572539.304328:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.306205:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.306324:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360256256 00000100:00100000:2.0:1788572539.306334:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360256256 00010000:00010000:2.0:1788572539.306376:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 77 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.306411:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 901 00000100:00100000:2.0:1788572539.306420:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:2349:x1875453360256256:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572539.306435:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572539.306443:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 77 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.306455:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 77 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.306489:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb16149d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef347 expref: 77 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.306507:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x70:0x0].0x0 (ffff9627e7a95600) 00010000:00010000:2.0:1788572539.306512:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.306523:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.306532:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.306562:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572539.306616:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2349:x1875453360256256:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 197us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.306624:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 901 00010000:00010000:3.0:1788572539.307278:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.307289:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572539.307493:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970151 00000001:00080000:3.0:1788572539.307662:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970152 00000001:00080000:3.0:1788572539.307711:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970153 00000100:00100000:3.0:1788572539.307800:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382333824:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572539.307868:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.307902:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.308734:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382333824 00000100:00100000:2.0:1788572539.308744:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382333824 00000100:00100000:2.0:1788572539.308789:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1788572539.308798:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382333824:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1788572539.308900:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382333824:12345-0@lo:1000:mdt00_005.0 Request processed in 102us (1031us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.308907:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:3.0:1788572539.309185:0:23058:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967531 gen 2 00000004:00080000:3.0:1788572539.309192:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967529 committed transno 4294967531 00000004:00080000:3.0:1788572539.309211:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967531 committed transno 4294967531 00000100:00100000:3.0:1788572539.309602:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382333824:0@lo:1000:mdt00_005.0 00000001:00080000:3.0:1788572539.310795:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970154 00000100:00100000:3.0:1788572539.310908:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382333952:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572539.311170:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.311250:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.311590:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382333952 00000100:00100000:2.0:1788572539.311599:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382333952 00000100:00100000:2.0:1788572539.311701:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1788572539.311713:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382333952:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1788572539.311968:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382333952:12345-0@lo:1000:mdt00_005.0 Request processed in 257us (797us total) trans 4294967535 rc 0/0 00000100:00100000:2.0:1788572539.311977:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:3.0:1788572539.312070:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382333952:0@lo:1000:mdt00_005.0 00010000:00010000:3.0:1788572539.312111:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312122:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312130:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312136:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.312140:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312148:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312164:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x70:0x0].0x0 (ffff9627e7a95600) 00010000:00010000:3.0:1788572539.312166:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572539.312172:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312178:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16155a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x70:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312209:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312226:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312233:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312238:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.312242:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312248:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312256:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x79:0x0].0x0 (ffff9627d599a200) 00010000:00010000:3.0:1788572539.312259:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572539.312262:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 2/0,0 mode: --/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312269:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935400/0x85ace67feb15ea7a lrc: 0/0,0 mode: --/EX res: [0x200000401:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312289:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312301:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312311:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312318:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312324:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312330:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312334:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.312338:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312344:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312351:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x79:0x0].0x0 (ffff9627d599a200) 00010000:00010000:3.0:1788572539.312353:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.312356:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312362:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161561 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x79:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312370:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312377:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312383:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312388:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.312391:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312397:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312404:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x79:0x0].0x0 (ffff9627d599a200) 00010000:00010000:3.0:1788572539.312406:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572539.312409:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.312414:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161553 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.312581:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35885:x1875453360255872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 13344us (15177us total) trans 4294970155 rc 0/0 00000100:00100000:3.0:1788572539.312590:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3776 00000020:00080000:3.0:1788572539.312593:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1788572539.313495:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572539.313502:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572539.313606:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 167 count 101 00000040:00080000:1.0:1788572539.313615:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1788572539.313739:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ed880 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382334080:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572539.313817:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.316731:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.316831:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360256512 00000100:00100000:3.0:1788572539.316841:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360256512 00000100:00100000:3.0:1788572539.316904:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 847 00000100:00100000:3.0:1788572539.316915:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35885:x1875453360256512:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572539.317336:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x70:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572539.317377:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970156 00010000:00010000:3.0:1788572539.317422:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x70:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.317439:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.317451:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.317459:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.317465:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.317482:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.317498:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572539.317586:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35885:x1875453360256512:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 671us (864us total) trans 4294970157 rc 0/0 00000100:00100000:3.0:1788572539.317594:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 847 00000020:00080000:3.0:1788572539.317599:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788572539.320562:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572539.320576:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.320586:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572539.320591:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.320601:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.320617:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x70:0x0].0x0 (ffff9627d599a300) 00010000:00010000:1.0:1788572539.320620:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572539.320625:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.320632:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161568 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x70:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.320653:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572539.320655:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572539.320657:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572539.321513:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ed880 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382334080:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572539.327636:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.332437:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.340961:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.341080:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.345304:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.353493:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.356567:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.356655:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360257536 00000100:00100000:3.0:1788572539.356664:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360257536 00000100:00100000:3.0:1788572539.356718:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3783 00000100:00100000:3.0:1788572539.356728:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35889:x1875453360257536:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.356799:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356816:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356828:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356844:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356952:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356961:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356989:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.356997:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357030:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.357069:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7d:0x0]: returning size 3 00000004:00200000:3.0:1788572539.357074:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.357087:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357099:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357106:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357113:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357119:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.357123:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357131:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357142:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572539.357146:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.357151:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357157:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1615ae lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357169:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357175:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357180:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357185:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.357188:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357194:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357201:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x4e5eae2c (ffff9627d599a200) 00010000:00010000:3.0:1788572539.357204:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.357207:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357212:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615b5 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e5eae2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357370:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357378:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357383:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357389:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.357392:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357399:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357407:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:3.0:1788572539.357410:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.357413:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.357419:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.357508:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35889:x1875453360257536:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 780us (949us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.357517:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3783 00000100:00100000:3.0:1788572539.363415:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.365576:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.365651:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360257920 00000100:00100000:1.0:1788572539.365659:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360257920 00000100:00100000:1.0:1788572539.365719:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3785 00000100:00100000:1.0:1788572539.365728:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35890:x1875453360257920:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572539.365761:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbf:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572539.365872:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35890:x1875453360257920:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 145us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.365880:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3785 00000100:00100000:1.0:1788572539.371861:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.371903:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.372044:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360258176 00000100:00100000:3.0:1788572539.372056:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360258176 00000100:00100000:3.0:1788572539.372119:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360258304 00000100:00100000:3.0:1788572539.372123:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360258304 00000100:00100000:3.0:1788572539.372153:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3786 00000100:00100000:3.0:1788572539.372173:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35889:x1875453360258176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.372195:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.372215:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615d8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef386 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372317:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372332:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372344:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.372356:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7d:0x0]: returning size 3 00010000:00010000:3.0:1788572539.372452:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372478:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372485:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372491:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.372494:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372501:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372514:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:3.0:1788572539.372518:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572539.372522:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 2/0,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372529:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1615df lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372574:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615d8 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef386 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372584:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572539.372586:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572539.372590:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1615d8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef386 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.372598:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:3.0:1788572539.372657:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35889:x1875453360258176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 486us (808us total) trans 4294970160 rc 301/301 00000100:00100000:3.0:1788572539.372664:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3786 00000100:00100000:3.0:1788572539.372683:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3787 00000100:00100000:3.0:1788572539.372689:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35888:x1875453360258304:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572539.372709:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x7e:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572539.372778:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35888:x1875453360258304:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 89us (874us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.372785:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3787 00000100:00100000:2.0:1788572539.386114:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.386379:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360258432 00000100:00100000:1.0:1788572539.386390:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360258432 00000100:00100000:1.0:1788572539.386442:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3788 00000100:00100000:1.0:1788572539.386452:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35890:x1875453360258432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.386490:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.386514:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1615ed lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef38d expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386586:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386603:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386616:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386631:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386637:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386649:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386980:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161602 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.386991:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161602 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387000:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161602 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572539.387059:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:1.0:1788572539.387063:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572539.387084:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161602 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387096:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387102:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387109:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387115:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.387119:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387127:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387141:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572539.387143:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.387149:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387155:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1615f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387178:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387185:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387191:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387197:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.387200:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387208:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387215:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.387226:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.387229:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387236:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1615fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387274:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1615ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef38d expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387288:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161602 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef38d expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.387296:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572539.387298:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572539.387300:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:1.0:1788572539.387364:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35890:x1875453360258432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 913us (1260us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.387372:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3788 00000100:00100000:3.0:1788572539.392914:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.393353:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360258560 00000100:00100000:2.0:1788572539.393364:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360258560 00000100:00100000:2.0:1788572539.393426:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3789 00000100:00100000:2.0:1788572539.393437:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35889:x1875453360258560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572539.393561:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x7d:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572539.393673:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35889:x1875453360258560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 237us (767us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.393681:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3789 00000100:00100000:3.0:1788572539.413542:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000004:00080000:1.0:1788572539.415643:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x40:0x0], index 1: 0 00010000:00010000:1.0:1788572539.415867:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161610 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.415884:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161610 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.415898:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161610 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.415984:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161609 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef394 expref: 87 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.416001:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161610 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef394 expref: 86 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.416056:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.416058:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572539.416062:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:1.0:1788572539.416142:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35890:x1875453360258816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2338us (2609us total) trans 4294970161 rc 0/0 00000100:00100000:1.0:1788572539.416151:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3790 00000100:00100000:1.0:1788572539.427056:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360258944 00000100:00100000:1.0:1788572539.427073:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360258944 00000100:00100000:1.0:1788572539.427134:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3791 00000100:00100000:1.0:1788572539.427144:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35888:x1875453360258944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.427179:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.427201:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16161e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef39b expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427287:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427302:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427314:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427330:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427338:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427346:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427674:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161633 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427685:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161633 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427692:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161633 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572539.427722:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:1.0:1788572539.427727:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572539.427750:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161633 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427763:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427770:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427777:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427783:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.427788:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427796:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427810:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572539.427813:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.427819:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427825:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb161625 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427837:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427844:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427849:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427855:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.427857:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427864:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427871:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d0ee2100) 00010000:00010000:1.0:1788572539.427874:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.427876:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427881:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16162c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427919:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16161e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef39b expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427934:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161633 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef39b expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.427940:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572539.427942:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572539.427945:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:1.0:1788572539.428056:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35888:x1875453360258944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 913us (6346us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.428064:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3791 00000100:00100000:1.0:1788572539.430084:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360259072 00000100:00100000:1.0:1788572539.430100:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360259072 00000100:00100000:1.0:1788572539.430182:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3792 00000100:00100000:1.0:1788572539.430194:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35890:x1875453360259072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.430215:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.430245:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb16163a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3a2 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430310:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430324:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430334:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430348:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430355:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430361:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430649:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16164f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430659:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16164f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430667:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16164f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572539.430692:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:1.0:1788572539.430695:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572539.430713:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16164f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430724:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430730:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430736:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430747:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.430751:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430759:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430771:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572539.430774:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.430779:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430786:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161641 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430799:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430805:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430811:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430816:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.430818:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430823:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430833:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627d0ee2000) 00010000:00010000:1.0:1788572539.430835:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.430838:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430843:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161648 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430874:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb16163a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3a2 expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430888:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16164f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3a2 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.430895:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572539.430897:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572539.430900:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fe00, rc 0) 00000100:00100000:1.0:1788572539.430961:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35890:x1875453360259072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 769us (2866us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.430969:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3792 00000100:00100000:1.0:1788572539.437690:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360259328 00000100:00100000:1.0:1788572539.437699:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360259328 00000100:00100000:1.0:1788572539.437762:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3793 00000100:00100000:1.0:1788572539.437773:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35890:x1875453360259328:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.437835:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572539.437862:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161610 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef394 expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.437886:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.437917:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16164f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3a2 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.437925:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572539.437928:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572539.437964:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.437978:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 159 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.437987:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.438067:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 159 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.438081:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 159 pid: 18263 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.438095:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382334208:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572539.438190:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f840 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.442100:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360259584 00000100:00100000:2.0:1788572539.442112:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360259584 00000100:00100000:2.0:1788572539.442168:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3794 00000100:00100000:2.0:1788572539.442177:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35888:x1875453360259584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.442198:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.442201:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572539.442226:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb161633 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef39b expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.442247:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572539.442253:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572539.442271:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16165d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3a9 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572539.442801:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x41:0x0], index 0: 0 00010000:00010000:2.0:1788572539.443449:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161664 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.443483:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161664 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.443496:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161664 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.443561:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16165d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3a9 expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.443579:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161664 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3a9 expref: 88 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.446708:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627c2abc400) 00010000:00010000:2.0:1788572539.446713:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572539.446717:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572539.446814:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35888:x1875453360259584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4632us (4765us total) trans 4294970162 rc 0/0 00000100:00100000:2.0:1788572539.446824:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3794 00000100:00100000:1.0:1788572539.448712:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360259712 00000100:00100000:1.0:1788572539.448721:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360259712 00000100:00100000:1.0:1788572539.448771:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3795 00000100:00100000:1.0:1788572539.448780:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35889:x1875453360259712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.448809:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.448829:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161672 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3b0 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.448896:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.448910:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.448920:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.448933:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.448949:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.448956:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449282:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161687 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449291:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161687 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449298:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161687 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572539.449322:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:1.0:1788572539.449325:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572539.449341:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161687 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449352:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449359:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449366:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449372:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.449376:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449383:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449394:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572539.449396:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.449400:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449406:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb161679 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449417:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449423:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449433:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449438:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.449441:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449446:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449452:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.449455:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.449457:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449462:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161680 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449503:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb161672 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3b0 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449514:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161687 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3b0 expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.449521:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572539.449523:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572539.449525:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:1.0:1788572539.449570:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35889:x1875453360259712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 791us (938us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.449581:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3795 00000100:00100000:1.0:1788572539.456540:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382334208:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.456573:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.457695:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360260096 00000100:00100000:1.0:1788572539.457703:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360260096 00000100:00100000:1.0:1788572539.457768:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3796 00000100:00100000:1.0:1788572539.457779:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35888:x1875453360260096:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.457842:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572539.457874:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161664 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3a9 expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.457893:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627c2abc400) 00010000:00010000:1.0:1788572539.457909:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb161687 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3b0 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.457918:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572539.457921:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572539.457948:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.457962:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.457971:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.457990:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.457996:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458001:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458260:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458269:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458276:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458281:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.458285:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458292:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458301:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572539.458303:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.458307:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458314:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16168e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458322:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458329:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458335:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458340:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.458342:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458348:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458356:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0xf1b96b96 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.458358:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.458361:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.458366:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb161695 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x76:0x0].0xf1b96b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.458445:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35888:x1875453360260096:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 667us (812us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572539.458453:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3796 00000100:00100000:3.0:1788572539.459380:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360259840 00000100:00100000:3.0:1788572539.459392:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360259840 00010000:00010000:3.0:1788572539.459445:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 160 pid: 18263 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.459490:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 902 00000100:00100000:3.0:1788572539.459499:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+160:2350:x1875453360259840:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.459511:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.459517:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 160 pid: 18263 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.459528:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 160 pid: 18263 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.459543:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb15da12 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddec expref: 160 pid: 18263 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.459555:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572539.459559:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.459569:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.459576:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.459610:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.459650:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:2350:x1875453360259840:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 153us (2060us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.459660:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 902 00010000:00010000:2.0:1788572539.459723:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459732:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459754:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459763:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459777:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459975:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459983:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459989:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.459994:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.459998:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460054:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460063:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572539.460065:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572539.460069:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460075:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161656 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460083:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460088:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460094:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460099:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.460101:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460105:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460111:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x216b58e9 (ffff9627c2abc700) 00010000:00010000:2.0:1788572539.460113:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572539.460115:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.460120:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16169c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x216b58e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.460174:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35890:x1875453360259328:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 22403us (22576us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572539.460181:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3793 00000100:00080000:1.0:1788572539.461676:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572539.468416:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.468463:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.469366:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360260224 00000100:00100000:3.0:1788572539.469380:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360260224 00000100:00100000:3.0:1788572539.469481:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360260608 00000100:00100000:3.0:1788572539.469486:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360260608 00000100:00100000:2.0:1788572539.469512:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360260352 00000100:00100000:2.0:1788572539.469516:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360260352 00000100:00100000:1.0:1788572539.469563:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3797 00000100:00100000:1.0:1788572539.469574:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35888:x1875453360260224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.469596:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.469622:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616a3 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3b7 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.469661:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1616aa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.469675:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1616aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x73:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.469687:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1616aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x73:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.469794:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616a3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3b7 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.469814:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1616aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3b7 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.469823:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.469826:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572539.469829:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fe00, rc 0) 00000100:00100000:1.0:1788572539.469891:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35888:x1875453360260224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 320us (1486us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.469899:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3797 00000100:00100000:1.0:1788572539.469920:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3798 00000100:00100000:1.0:1788572539.469927:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35889:x1875453360260608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.469953:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.469963:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616b1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3c5 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572539.470405:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x41:0x0], index 1: 0 00010000:00010000:1.0:1788572539.470627:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1616b8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470639:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1616b8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470649:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1616b8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470708:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616b1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3c5 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470723:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1616b8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3c5 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470731:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572539.470734:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572539.470737:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:1.0:1788572539.470782:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35889:x1875453360260608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 855us (2318us total) trans 4294970163 rc 0/0 00000100:00100000:1.0:1788572539.470788:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3798 00000100:00100000:1.0:1788572539.470811:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3799 00000100:00100000:1.0:1788572539.470820:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35890:x1875453360260352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.470830:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.470841:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616c6 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3be expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470860:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 3/1,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470868:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470875:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470938:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616c6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470952:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.470958:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:1.0:1788572539.470961:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572539.470964:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:1.0:1788572539.470985:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35890:x1875453360260352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 167us (2001us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.470992:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3799 00000100:00100000:1.0:1788572539.488945:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360260992 00000100:00100000:1.0:1788572539.488956:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360260992 00000100:00100000:1.0:1788572539.489041:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360261120 00000100:00100000:1.0:1788572539.489045:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360261120 00000100:00100000:1.0:1788572539.489063:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360261248 00000100:00100000:1.0:1788572539.489065:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360261248 00000100:00100000:1.0:1788572539.489082:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3800 00000100:00100000:1.0:1788572539.489090:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35888:x1875453360260992:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.489188:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.489207:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.489232:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572539.489661:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].53, 88 off12376 00000004:00080000:1.0:1788572539.489670:0:5839:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 887558665485ns, next at 888558663380ns, rc = 0 00000004:00010000:1.0:1788572539.489765:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000063186c44 reply state 000000000424e0c9 transno 4294970164 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.489775:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572539.489861:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35888:x1875453360260992:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 770us (1074us total) trans 4294970164 rc 0/0 00000100:00100000:1.0:1788572539.489869:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3800 00000100:00100000:1.0:1788572539.489890:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3801 00000100:00100000:1.0:1788572539.489897:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35890:x1875453360261120:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.489929:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,1 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.489939:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.489952:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572539.490169:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].50, 88 off12184 00000004:00080000:1.0:1788572539.490175:0:5839:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 887559170431ns, next at 888559169136ns, rc = 0 00000004:00010000:1.0:1788572539.490220:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000099786fa3 reply state 00000000d083c91e transno 4294970165 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.490226:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572539.490259:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35890:x1875453360261120:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 362us (1412us total) trans 4294970165 rc 0/0 00000100:00100000:1.0:1788572539.490265:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3801 00000100:00100000:1.0:1788572539.490279:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3802 00000100:00100000:1.0:1788572539.490285:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35889:x1875453360261248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.490294:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.490322:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616e2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3cc expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490370:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490380:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490387:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490399:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490406:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490412:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490612:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1616f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490621:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1616f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490629:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1616f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572539.490648:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:1.0:1788572539.490651:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572539.490662:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1616f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490672:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490679:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490685:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490692:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.490697:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490704:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490717:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572539.490720:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.490725:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490730:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1616e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490742:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490748:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490754:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490759:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.490762:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490767:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490774:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa7896503 (ffff9627d0ee2200) 00010000:00010000:1.0:1788572539.490777:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.490780:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490791:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1616f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa7896503 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490818:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616e2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3cc expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490830:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1616f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3cc expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.490838:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572539.490840:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572539.490842:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:1.0:1788572539.490867:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35889:x1875453360261248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 583us (1995us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.490872:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3802 00010000:00010000:2.0:1788572539.496801:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.496821:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.496828:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.496834:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.496839:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.496848:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.496866:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:2.0:1788572539.496869:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572539.496874:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.496880:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1616d4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.501051:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360261376 00000100:00100000:1.0:1788572539.501062:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360261376 00000100:00100000:1.0:1788572539.501127:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3803 00000100:00100000:1.0:1788572539.501137:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35890:x1875453360261376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.501169:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.501193:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 2/0,0 mode: --/CR res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3d3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.501256:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.501265:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.501282:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.505988:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.506103:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.508729:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360261504 00000100:00100000:1.0:1788572539.508744:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360261504 00000100:00100000:1.0:1788572539.508799:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360261760 00000100:00100000:1.0:1788572539.508880:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360261760 00000100:00100000:1.0:1788572539.508916:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3804 00000100:00100000:1.0:1788572539.508926:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35888:x1875453360261504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.508974:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572539.509001:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3da expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509085:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 97 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509099:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 97 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509117:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.509120:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572539.509124:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e000, rc 0) 00000100:00100000:1.0:1788572539.509183:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35888:x1875453360261504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 258us (2730us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.509189:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3804 00000100:00100000:1.0:1788572539.509207:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3805 00000100:00100000:1.0:1788572539.509215:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35889:x1875453360261760:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.509268:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572539.509550:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1616b8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3c5 expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509569:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572539.509586:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1616f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3cc expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509596:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572539.509600:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572539.509616:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16170c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509626:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16170c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509635:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16170c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509651:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161713 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x76:0x0].0xf98cdcf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509658:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161713 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x76:0x0].0xf98cdcf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.509665:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161713 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x76:0x0].0xf98cdcf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.517106:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.517120:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.517136:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.517155:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:3.0:1788572539.517158:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1788572539.517163:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 2/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.517171:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1616db lrc: 0/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.518383:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.519970:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360261888 00000100:00100000:1.0:1788572539.519981:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360261888 00000100:00100000:1.0:1788572539.520054:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 953 00000100:00100000:1.0:1788572539.520065:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360261888:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.520621:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360261888:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 556us (2244us total) trans 4294967536 rc 0/0 00000100:00100000:1.0:1788572539.520630:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 953 00000100:00100000:2.0:1788572539.531229:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.533446:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360262144 00000100:00100000:2.0:1788572539.533454:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360262144 00000100:00100000:2.0:1788572539.533522:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3806 00000100:00100000:2.0:1788572539.533532:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35889:x1875453360262144:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.533552:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.533577:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16171a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3e1 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.533613:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.533628:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.533642:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.533770:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16171a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.533789:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 94 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.533799:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:2.0:1788572539.533802:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572539.533806:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00000100:00100000:2.0:1788572539.533864:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35889:x1875453360262144:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 334us (504us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.533871:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3806 00000100:00100000:1.0:1788572539.535205:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.539794:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360262656 00000100:00100000:2.0:1788572539.539805:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360262656 00000100:00100000:2.0:1788572539.539867:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3807 00000100:00100000:2.0:1788572539.539877:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35890:x1875453360262656:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572539.539999:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35890:x1875453360262656:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 123us (306us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.540044:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3807 00000100:00100000:1.0:1788572539.546110:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.546679:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360262912 00000100:00100000:2.0:1788572539.546692:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360262912 00000100:00100000:2.0:1788572539.546774:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3808 00000100:00100000:2.0:1788572539.546785:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35889:x1875453360262912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.546927:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.546945:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.546957:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.547492:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360263168 00000100:00100000:1.0:1788572539.547497:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360263168 00000100:00100000:1.0:1788572539.547535:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 849 00000100:00100000:1.0:1788572539.547544:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35888:x1875453360263168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.547933:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35888:x1875453360263168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 388us (620us total) trans 4294970167 rc 0/0 00000100:00100000:1.0:1788572539.547951:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 849 00000040:00080000:2.0:1788572539.548304:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].51, 88 off12272 00000004:00010000:2.0:1788572539.548370:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001f72de6a reply state 000000009e291053 transno 4294970168 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.548380:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572539.548447:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35889:x1875453360262912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1663us (2345us total) trans 4294970168 rc 0/0 00000100:00100000:2.0:1788572539.548454:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3808 00000100:00100000:3.0:1788572539.551704:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.551925:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360263424 00000100:00100000:2.0:1788572539.551936:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360263424 00000100:00100000:2.0:1788572539.551994:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360263552 00000100:00100000:2.0:1788572539.551997:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360263552 00000100:00100000:2.0:1788572539.552055:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3809 00000100:00100000:2.0:1788572539.552064:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35888:x1875453360263424:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572539.552184:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35888:x1875453360263424:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 121us (1106us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.552192:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3809 00000100:00100000:2.0:1788572539.552212:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3810 00000100:00100000:2.0:1788572539.552218:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35890:x1875453360263552:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.552230:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.552250:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16172f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3e8 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.552306:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161736 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.552319:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161736 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.552330:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161736 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572539.552367:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572539.552385:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161736 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.552410:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16172f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e8 expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.552427:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161736 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e8 expref: 99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.552435:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:2.0:1788572539.552438:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572539.552441:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572539.552463:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35890:x1875453360263552:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 245us (761us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.552480:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3810 00010000:00010000:2.0:1788572539.568039:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.568060:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.568068:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.568073:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.568077:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.568085:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.568102:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:2.0:1788572539.568106:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572539.568111:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.568118:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161728 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.569784:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.569831:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.569973:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360263808 00000100:00100000:2.0:1788572539.569984:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360263808 00000100:00100000:2.0:1788572539.570054:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 955 00000100:00100000:2.0:1788572539.570065:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360263808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.570082:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.570103:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3f6 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572539.570138:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x40:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737b, blocks: 0x1 00010000:00010000:2.0:1788572539.570158:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=887s v=(0 0) 00010000:00010000:2.0:1788572539.570164:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572539.570173:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.570191:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.570202:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:2.0:1788572539.570261:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360263808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 198us (428us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.570269:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 955 00000100:00100000:3.0:1788572539.584325:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.584505:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360263936 00000100:00100000:2.0:1788572539.584516:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360263936 00000100:00100000:2.0:1788572539.584578:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3812 00000100:00100000:2.0:1788572539.584588:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35889:x1875453360263936:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.584614:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.584636:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef3fd expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.584699:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.584715:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.584737:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:2.0:1788572539.584741:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572539.584745:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572539.584803:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35889:x1875453360263936:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 217us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.584810:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3812 00000100:00100000:1.0:1788572539.585987:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.586099:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360264192 00000100:00100000:3.0:1788572539.586107:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360264192 00000100:00100000:3.0:1788572539.586164:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3813 00000100:00100000:3.0:1788572539.586174:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35890:x1875453360264192:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.586273:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35890:x1875453360264192:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 100us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.586281:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3813 00000100:00100000:3.0:1788572539.594914:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360264320 00000100:00100000:3.0:1788572539.594925:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360264320 00000100:00100000:3.0:1788572539.594988:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360264448 00000100:00100000:3.0:1788572539.594991:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360264448 00000100:00100000:3.0:1788572539.595048:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 956 00000100:00100000:3.0:1788572539.595060:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360264320:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.595079:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.595101:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef404 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572539.595129:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x41:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737b, blocks: 0x1 00010000:00010000:3.0:1788572539.595153:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=887s v=(0 0) 00010000:00010000:3.0:1788572539.595330:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00000100:00100000:2.0:1788572539.595341:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 957 00010000:00010000:3.0:1788572539.595343:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.595353:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:35890:x1875453360264448:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.595361:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.595363:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.595372:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef40b expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.595382:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53000, rc 0) 00010000:00010000:2.0:1788572539.595391:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.595398:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572539.595401:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572539.595405:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.595436:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.595437:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360264320:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 381us (671us total) trans 0 rc 0/0 00010000:00010000:2.0:1788572539.595451:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 13 pid: 6681 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.595547:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382334336:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572539.595608:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b86c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572539.606327:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 956 00000100:00100000:1.0:1788572539.609256:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.609360:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.609557:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360264576 00000100:00100000:2.0:1788572539.609569:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360264576 00000100:00100000:2.0:1788572539.609640:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 958 00000100:00100000:2.0:1788572539.609651:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360264576:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572539.610202:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360264576:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 550us (954us total) trans 4294967537 rc 0/0 00000100:00100000:2.0:1788572539.610484:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 958 00000100:00100000:3.0:1788572539.626135:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.626529:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360264960 00000100:00100000:1.0:1788572539.626540:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360264960 00010000:00010000:1.0:1788572539.626593:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 15 pid: 6681 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.626649:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 903 00000100:00100000:1.0:1788572539.626659:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360264960:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.626675:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1788572539.626683:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382334336:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.626698:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 15 pid: 6681 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.626716:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 15 pid: 6681 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.632439:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 15 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.632482:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04600, rc 0) 00000100:00100000:2.0:1788572539.632564:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35890:x1875453360264448:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 37213us (37737us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.632573:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 957 00010000:00010000:1.0:1788572539.635288:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb16173d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3f6 expref: 14 pid: 6681 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.635319:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.635325:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.635336:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.635347:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.635836:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.635947:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360265216 00000100:00100000:3.0:1788572539.635957:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360265216 00000100:00100000:3.0:1788572539.636037:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 850 00000100:00100000:3.0:1788572539.636046:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35889:x1875453360265216:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572539.637928:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35889:x1875453360265216:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1871us (2082us total) trans 4294970169 rc 0/0 00000100:00100000:2.0:1788572539.637941:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 850 00000100:00100000:1.0:1788572539.638571:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.639050:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360265344 00000100:00100000:3.0:1788572539.639059:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360265344 00000100:00100000:3.0:1788572539.639103:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 959 00000100:00100000:3.0:1788572539.639112:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35888:x1875453360265344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.639128:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.639148:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef412 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.639182:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.639191:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.639195:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.639200:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 14 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.639248:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.639263:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 14 pid: 10416 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.639278:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382334592:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.639351:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.645369:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.645430:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.645991:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360265472 00000100:00100000:2.0:1788572539.646001:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360265472 00010000:00010000:2.0:1788572539.646091:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 15 pid: 10416 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.646102:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382334592:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572539.646128:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 904 00010000:00010000:3.0:1788572539.646133:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 15 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.646136:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360265472:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572539.646151:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.646151:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00010000:00010000:2.0:1788572539.646189:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 15 pid: 10416 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.646202:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 15 pid: 10416 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.646206:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35888:x1875453360265344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 7096us (7638us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.646226:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 959 00010000:00010000:2.0:1788572539.646233:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000da036d4a/0x85ace67feb161759 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef404 expref: 15 pid: 10416 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.646247:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572539.646252:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572539.646260:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.646270:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.646308:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.646329:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382334720:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572539.646383:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8540 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.651876:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.651958:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.651991:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360265856 00000100:00100000:2.0:1788572539.652000:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360265856 00000100:00100000:2.0:1788572539.652078:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3815 00000100:00100000:2.0:1788572539.652088:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35890:x1875453360265856:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.652189:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360265728 00000100:00100000:1.0:1788572539.652194:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360265728 00002000:00100000:1.0:1788572539.652242:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x41:0x0] (0->2) req@ffff9627dfdbdc00 x1875453360265728/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/0 e 0 to 0 dl 1788572550 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572539.652252:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbdc00 with x1875453360265728 ext(0->2) 00002000:00010000:1.0:1788572539.652257:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627dfdbdc00 00000100:00100000:1.0:1788572539.652276:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 224 00000100:00100000:1.0:1788572539.652282:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360265728:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.652570:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382334720:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572539.652593:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.657352:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.659650:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360265472:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 13501us (14281us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.659665:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 904 00000100:00100000:1.0:1788572539.660159:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.666694:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.666959:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360266368 00000100:00100000:3.0:1788572539.666970:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360266368 00000100:00100000:3.0:1788572539.667055:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3818 00000100:00100000:3.0:1788572539.667066:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35889:x1875453360266368:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.667090:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.667114:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16176e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef419 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.667666:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161775 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.667685:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161775 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.667697:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161775 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.667736:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.667754:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161775 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.667785:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16176e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef419 expref: 105 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.667803:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161775 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef419 expref: 104 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.667811:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:3.0:1788572539.667814:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572539.667818:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:3.0:1788572539.667890:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+104:35889:x1875453360266368:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 825us (1205us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.667900:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3818 00000100:00100000:2.0:1788572539.677810:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.678252:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360266496 00000100:00100000:3.0:1788572539.678262:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360266496 00000100:00100000:3.0:1788572539.678313:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 960 00000100:00100000:3.0:1788572539.678322:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360266496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.678339:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.678357:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16177c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef420 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572539.678382:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x41:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737b, blocks: 0x1 00010000:00010000:3.0:1788572539.678399:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=887s v=(0 0) 00010000:00010000:3.0:1788572539.678403:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.678412:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16177c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef420 expref: 16 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.678430:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16177c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef420 expref: 16 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.678443:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:3.0:1788572539.678505:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360266496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 184us (706us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.678512:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 960 00000100:00100000:3.0:1788572539.682851:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.682916:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360266752 00000100:00100000:2.0:1788572539.682929:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360266752 00000100:00100000:2.0:1788572539.683080:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3819 00000100:00100000:2.0:1788572539.683091:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+104:35889:x1875453360266752:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572539.683889:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+104:35889:x1875453360266752:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 797us (1044us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.683898:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3819 00000100:00100000:2.0:1788572539.689737:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.689916:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360266880 00000100:00100000:3.0:1788572539.689927:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360266880 00000100:00100000:3.0:1788572539.689994:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 961 00000100:00100000:3.0:1788572539.690018:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7663050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35889:x1875453360266880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.690039:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.690061:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef427 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.690091:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16177c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef420 expref: 17 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.690100:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.690103:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572539.690109:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.690144:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16177c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef420 expref: 17 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.690159:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16177c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef420 expref: 17 pid: 10416 timeout: 987 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.690174:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382334848:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.690269:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.703771:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.703864:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382334848:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.703893:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 18 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.703916:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:3.0:1788572539.703973:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7663050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:35889:x1875453360266880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 13972us (14249us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.703980:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 961 00000100:00100000:2.0:1788572539.717331:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.718367:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360267136 00000100:00100000:1.0:1788572539.718379:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360267136 00002000:00100000:1.0:1788572539.718433:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x41:0x0] (0->2) req@ffff9627dfdbdf80 x1875453360267136/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/0 e 0 to 0 dl 1788572550 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572539.718441:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbdf80 with x1875453360267136 ext(0->2) 00002000:00010000:1.0:1788572539.718447:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbdf80 00000100:00100000:1.0:1788572539.718486:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 225 00000100:00100000:1.0:1788572539.718496:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360267136:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572539.718883:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360267136:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 387us (1559us total) trans 4294967539 rc 0/0 00000100:00100000:1.0:1788572539.718891:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 225 00002000:00100000:1.0:1788572539.718899:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x41:0x0] (0->2) req@ffff9627dfdbdf80 x1875453360267136/t4294967539(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:200/0 lens 496/448 e 0 to 0 dl 1788572550 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572539.718907:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbdf80 with x1875453360267136 ext(0->2) 00002000:00010000:1.0:1788572539.718910:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbdf80 00000100:00100000:3.0:1788572539.725560:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.725649:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360267264 00000100:00100000:2.0:1788572539.725659:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360267264 00000100:00100000:2.0:1788572539.725722:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3820 00000100:00100000:2.0:1788572539.725732:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+104:35889:x1875453360267264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572539.725871:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+104:35889:x1875453360267264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 140us (322us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.725879:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3820 00000100:00100000:2.0:1788572539.731690:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35890:x1875453360265856:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 79591us (79719us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.731706:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3815 00000001:00080000:2.0:1788572539.733038:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970166 is committed 00000020:00080000:2.0:1788572539.733062:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ff57a0c0 refcount 2 committed 0 result 0 batchid 91 00000020:00080000:2.0:1788572539.733067:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c1bf4b00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff3fe0c0 00000020:00080000:2.0:1788572539.733071:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c1bf4500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff091400 00000020:00080000:2.0:1788572539.733090:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 90 batchid updated 00000001:00080000:2.0:1788572539.733100:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970167 is committed 00000004:00080000:2.0:1788572539.733110:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 33 committed 00000004:00080000:2.0:1788572539.733135:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 38 committed 00000001:00080000:2.0:1788572539.733168:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970168 is committed 00000001:00080000:2.0:1788572539.733185:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970169 is committed 00000020:00080000:2.0:1788572539.733197:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2eb56c0 refcount 2 committed 0 result 0 batchid 93 00000020:00080000:2.0:1788572539.733200:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8adf00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2eb5000 00000020:00080000:2.0:1788572539.733203:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627dfaba800 00000020:00080000:2.0:1788572539.733229:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c83f10c0 refcount 2 committed 0 result 0 batchid 92 00000020:00080000:2.0:1788572539.733232:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad400 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d160c480 00000020:00080000:2.0:1788572539.733235:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9628009dc800 00000100:00100000:2.0:1788572539.733418:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572539.733437:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382335360:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572539.733516:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572539.733957:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 963 00000100:00100000:2.0:1788572539.733967:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382335232:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572539.734188:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382335616:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572539.734292:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.734309:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788572539.734315:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382335744:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572539.734336:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.734352:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788572539.734359:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382335488:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572539.734382:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.734400:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382335616 00000100:00100000:3.0:1788572539.734404:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382335616 00000100:00100000:3.0:1788572539.734423:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382335744 00000100:00100000:3.0:1788572539.734426:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382335744 00000100:00100000:3.0:1788572539.734434:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382335488 00000100:00100000:3.0:1788572539.734437:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382335488 00000100:00100000:3.0:1788572539.734447:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 964 00000100:00100000:3.0:1788572539.734453:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382335104:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572539.734572:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+12:3305:x1875453382335104:12345-0@lo:2:osp-syn-0-0.0 Request processed in 118us (767us total) trans 4294967543 rc 0/0 00000100:00100000:3.0:1788572539.734580:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 964 00000100:00100000:3.0:1788572539.734596:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 965 00000100:00100000:3.0:1788572539.734603:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3304:x1875453382335616:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572539.734706:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382335616:12345-0@lo:2:osp-syn-1-0.0 Request processed in 103us (413us total) trans 4294967540 rc 0/0 00000100:00100000:3.0:1788572539.734712:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 965 00000100:00100000:3.0:1788572539.734726:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 966 00000100:00100000:3.0:1788572539.734731:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3304:x1875453382335744:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1788572539.734798:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x40:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572539.734801:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x40:0x0] updating lvb mtime from disk: 0 -> 1788572539 00002000:00010000:3.0:1788572539.734803:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x40:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1788572539.734805:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x40:0x0] updating lvb ctime from disk: 1788572539 -> 1788572539 00002000:00010000:3.0:1788572539.734807:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x40:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1788572539.734833:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382335744:12345-0@lo:2:osp-syn-1-0.0 Request processed in 102us (495us total) trans 4294967541 rc 0/0 00000100:00100000:3.0:1788572539.734839:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 966 00000100:00100000:3.0:1788572539.734850:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 967 00000100:00100000:3.0:1788572539.734856:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382335488:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572539.734950:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382335488:12345-0@lo:2:osp-syn-1-0.0 Request processed in 94us (567us total) trans 4294967542 rc 0/0 00000100:00100000:3.0:1788572539.734957:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 967 00000004:00080000:3.0:1788572539.734999:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ff0aa680/2, rc 0, transno 4294967542 00000100:00100000:3.0:1788572539.735035:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382335488:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572539.735064:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfdbd180/2, rc 0, transno 4294967543 00000100:00100000:3.0:1788572539.735069:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382335104:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1788572539.735083:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ff0a9500/2, rc 0, transno 4294967541 00000100:00100000:3.0:1788572539.735089:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382335744:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572539.735098:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ff0a8e00/2, rc 0, transno 4294967540 00000100:00100000:3.0:1788572539.735101:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382335616:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572539.735294:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382335232:12345-0@lo:2:osp-syn-0-0.0 Request processed in 1330us (1546us total) trans 4294967542 rc 0/0 00000100:00100000:2.0:1788572539.735301:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 963 00000100:00100000:2.0:1788572539.736203:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.736500:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.736675:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360267392 00000100:00100000:2.0:1788572539.736683:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360267392 00000100:00100000:2.0:1788572539.736720:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360267520 00000100:00100000:2.0:1788572539.736722:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360267520 00000100:00100000:2.0:1788572539.736738:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 968 00000100:00100000:2.0:1788572539.736745:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360267392:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000004:00080000:3.0:1788572539.737433:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfdbed80/2, rc 0, transno 4294967542 00000100:00100000:3.0:1788572539.737441:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382335232:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1788572539.737460:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c9a5ad80/2, rc 0, transno 4294967541 00000100:00100000:3.0:1788572539.737462:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382335360:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572539.737528:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35888:x1875453360266240:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 76650us (77369us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.737537:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3817 00000100:00100000:2.0:1788572539.741542:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.786998:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.787518:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360267904 00000100:00100000:1.0:1788572539.787529:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360267904 00000100:00100000:1.0:1788572539.787590:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3821 00000100:00100000:1.0:1788572539.787602:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35888:x1875453360267904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.787659:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.787687:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16174b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3ef expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.787708:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.787714:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572539.787751:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.787766:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.787776:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572539.789082:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967538 is committed 00000001:00080000:3.0:1788572539.789095:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967539 is committed 00000001:00080000:3.0:1788572539.789112:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967535 is committed 00000001:00080000:3.0:1788572539.789118:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967542 is committed 00000100:00100000:2.0:1788572539.789663:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360267392:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 52916us (53462us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.789670:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 968 00010000:00010000:3.0:1788572539.791749:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.791763:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.791770:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.791775:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.791780:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.791787:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.791803:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572539.791806:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.791811:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.791817:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb16178a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.792572:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.793760:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360268160 00000100:00100000:1.0:1788572539.793773:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360268160 00000100:00100000:1.0:1788572539.794363:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3822 00000100:00100000:1.0:1788572539.794376:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35890:x1875453360268160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.794509:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.794610:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360268288 00000100:00100000:1.0:1788572539.794614:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360268288 00000100:00100000:1.0:1788572539.794651:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 971 00000100:00100000:1.0:1788572539.794658:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360268288:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572539.794845:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x41:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:1.0:1788572539.794848:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x41:0x0] updating lvb mtime from disk: 1788572539 -> 1788572520 00002000:00010000:1.0:1788572539.794851:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x41:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:1.0:1788572539.794853:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x41:0x0] updating lvb ctime from disk: 1788572539 -> 1788572524 00002000:00010000:1.0:1788572539.794856:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x41:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572539.795154:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360268288:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 494us (643us total) trans 4294967544 rc 0/0 00000100:00100000:1.0:1788572539.795163:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 971 00000100:00100000:3.0:1788572539.798486:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.798575:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360268544 00000100:00100000:2.0:1788572539.798586:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360268544 00000100:00100000:2.0:1788572539.798649:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3823 00000100:00100000:2.0:1788572539.798660:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35889:x1875453360268544:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.798722:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572539.798750:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161775 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef419 expref: 66 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.798768:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:2.0:1788572539.798773:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572539.798808:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.798823:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.798833:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572539.799059:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dcd1506c reply state 000000005a453824 transno 4294970172 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.799069:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572539.799177:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35889:x1875453360268544:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 518us (716us total) trans 4294970172 rc 0/0 00000100:00100000:2.0:1788572539.799185:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3823 00000100:00100000:2.0:1788572539.805997:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.806056:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.806076:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572539.806150:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.806164:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.806173:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.806178:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.806184:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.806191:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.806210:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:3.0:1788572539.806213:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.806218:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.806225:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161798 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806295:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806302:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806309:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806314:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.806317:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806322:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806332:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:1.0:1788572539.806334:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.806338:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.806344:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161791 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.812048:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360269312 00000100:00100000:2.0:1788572539.812058:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360269312 00000100:00100000:2.0:1788572539.812107:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3825 00000100:00100000:2.0:1788572539.812117:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35889:x1875453360269312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.812192:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.812210:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.812219:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.812259:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.812273:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 66 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.812287:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382336000:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572539.812371:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.813763:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572539.814040:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.814057:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.814076:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572539.814132:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360269440:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 181us (375us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572539.814139:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 906 00000100:00100000:1.0:1788572539.816058:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.816211:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360269696 00000100:00100000:2.0:1788572539.816232:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360269696 00000100:00100000:2.0:1788572539.816291:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3826 00000100:00100000:2.0:1788572539.816301:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35890:x1875453360269696:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.816365:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.816382:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.816392:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 4/0,1 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.816429:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.816442:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 67 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.816463:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382336128:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572539.816555:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec040300 going to sleep for 11 seconds 00010000:00010000:3.0:1788572539.817363:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1616aa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x73:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3b7 expref: 67 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.817384:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382335872:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.817408:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817419:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817425:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817456:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817474:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817482:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817488:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.817493:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817501:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817516:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572539.817520:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572539.817524:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.817530:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16179f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.817599:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35888:x1875453360268800:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10479us (11612us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572539.817609:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3824 00000100:00100000:1.0:1788572539.825173:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.825214:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.825339:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382336128:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.825366:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.825561:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360269824 00000100:00100000:0.0:1788572539.825568:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360269824 00010000:00010000:0.0:1788572539.825604:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 66 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.825647:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 907 00000100:00100000:0.0:1788572539.825657:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360269824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572539.825672:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572539.825679:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 66 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.825692:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 66 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.825713:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:0.0:1788572539.825718:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.825728:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.825740:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.825758:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572539.825811:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360269824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 155us (658us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572539.825819:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 907 00010000:00010000:2.0:1788572539.826689:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161721 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x74:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3e1 expref: 65 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.826707:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382336000:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572539.826724:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826732:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826737:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826765:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826772:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826778:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826784:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.826788:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826800:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826811:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:2.0:1788572539.826813:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572539.826817:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.826836:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1617a6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.826906:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35889:x1875453360269312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 14789us (15158us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572539.826922:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3825 00000100:00100000:0.0:1788572539.828054:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360269952 00000100:00100000:0.0:1788572539.828064:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360269952 00010000:00010000:0.0:1788572539.828100:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 64 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.828136:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 908 00000100:00100000:0.0:1788572539.828145:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360269952:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572539.828168:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572539.828175:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 64 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.828187:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 64 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.828205:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1616cd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x72:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3be expref: 64 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.828218:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:0.0:1788572539.828336:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.828347:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.828355:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.828388:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572539.828437:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2349:x1875453360269952:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 293us (3222us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572539.828445:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 908 00010000:00010000:3.0:1788572539.828882:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828891:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828929:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828942:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828950:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828960:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.828965:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828973:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828984:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:3.0:1788572539.828987:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572539.828990:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 2/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.828996:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1617ad lrc: 0/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.829083:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35890:x1875453360269696:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 12783us (13035us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572539.829092:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3826 00000100:00100000:3.0:1788572539.833654:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.833736:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270080 00000100:00100000:1.0:1788572539.833746:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270080 00000100:00100000:1.0:1788572539.833815:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 974 00000100:00100000:1.0:1788572539.833825:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360270080:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.833874:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.833895:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.833904:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.833909:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.833914:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.833963:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.833978:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 11 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.833993:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382336256:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572539.834117:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582aa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.838803:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.838909:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.845881:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.845916:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270464 00000100:00100000:1.0:1788572539.845926:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270464 00010000:00010000:1.0:1788572539.845978:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 12 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.846038:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 909 00000100:00100000:1.0:1788572539.846048:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360270464:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.846066:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.846093:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 12 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.846106:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 12 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.846134:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.846139:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.846145:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.846166:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.846191:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.846260:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360270464:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 212us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.846268:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 909 00010000:00010000:3.0:1788572539.846784:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161767 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef412 expref: 11 pid: 13272 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.846801:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382336256:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572539.846823:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846833:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846842:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846871:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846880:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846888:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846895:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572539.846899:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846908:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846922:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.846930:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb1617b4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.847040:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360270080:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 13216us (13396us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.847050:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 974 00000100:00100000:1.0:1788572539.850560:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.850601:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270720 00000100:00100000:1.0:1788572539.850611:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270720 00000100:00100000:1.0:1788572539.850666:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3827 00000100:00100000:1.0:1788572539.850676:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35888:x1875453360270720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.850760:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.850778:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.850792:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.851080:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.851092:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 63 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.851101:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.851139:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 63 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.851152:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 63 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.851168:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382336384:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572539.851326:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.854290:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.854355:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382336384:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.854383:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.854579:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270208 00000100:00100000:0.0:1788572539.854587:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270208 00000100:00100000:1.0:1788572539.854741:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270336 00000100:00100000:1.0:1788572539.854746:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270336 00000100:00100000:1.0:1788572539.854770:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 975 00000100:00100000:1.0:1788572539.854779:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360270208:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.854820:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.854839:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.854848:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.854852:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.854857:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.854892:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.854905:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 14 pid: 6681 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.854920:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382336512:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572539.855021:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.855040:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 976 00000100:00100000:1.0:1788572539.855048:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360270336:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572539.855069:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.855081:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.855088:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572539.855090:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.855093:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.855112:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.855122:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 14 pid: 6682 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.855132:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382336640:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.856147:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270848 00000100:00100000:3.0:1788572539.856293:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270848 00010000:00010000:3.0:1788572539.856332:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 64 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.856442:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582aa80 going to sleep for 11 seconds 00000100:00100000:3.0:1788572539.856625:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 910 00000100:00100000:3.0:1788572539.856634:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360270848:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572539.856648:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572539.856655:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 64 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.856668:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 64 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.856688:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb161705 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3da expref: 64 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.856704:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572539.856709:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.856719:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.856730:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.856760:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572539.856810:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2349:x1875453360270848:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 177us (2529us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.856818:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 910 00010000:00010000:1.0:1788572539.856878:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.856886:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572539.857110:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970173 00000001:00080000:1.0:1788572539.857299:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970174 00000001:00080000:1.0:1788572539.857347:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970175 00000100:00100000:1.0:1788572539.857427:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382336768:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572539.857494:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572539.857523:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.858812:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382336768 00000100:00100000:1.0:1788572539.858825:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382336768 00000100:00100000:1.0:1788572539.858923:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:1.0:1788572539.858933:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382336768:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572539.859067:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382336768:12345-0@lo:1000:mdt00_000.0 Request processed in 135us (1572us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.859075:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:1.0:1788572539.859121:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967543 gen 2 00000004:00080000:1.0:1788572539.859128:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967537 committed transno 4294967543 00000004:00080000:1.0:1788572539.859152:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967538 committed transno 4294967543 00000004:00080000:1.0:1788572539.859175:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c9a5ad80, transno 4294967541 00000004:00080000:1.0:1788572539.859183:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfdbed80, transno 4294967542 00000004:00080000:1.0:1788572539.859186:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfdbd180, transno 4294967543 00000100:00100000:1.0:1788572539.859194:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382336768:0@lo:1000:mdt00_000.0 00000004:00080000:2.0:1788572539.859641:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 3 done 00000001:00080000:1.0:1788572539.860138:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970176 00000100:00100000:1.0:1788572539.860838:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382336896:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572539.860893:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572539.860922:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e77343c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.863844:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382336896 00000100:00100000:2.0:1788572539.863870:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382336896 00000100:00100000:2.0:1788572539.863951:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:2.0:1788572539.863962:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382336896:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572539.864312:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382336896:12345-0@lo:1000:mdt00_000.0 Request processed in 352us (3418us total) trans 4294967545 rc 0/0 00000100:00100000:2.0:1788572539.864321:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:1.0:1788572539.864754:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382336896:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1788572539.864800:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864811:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864819:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864825:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.864830:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864838:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864854:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572539.864858:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572539.864863:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864870:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1617c2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x73:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864907:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864914:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864921:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864927:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.864930:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864950:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864958:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7e:0x0].0x0 (ffff9627d1866100) 00010000:00010000:1.0:1788572539.864961:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572539.864964:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864969:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb15ebfb lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.864990:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865022:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865032:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865039:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865045:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865051:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865055:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.865058:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865064:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865071:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7e:0x0].0x0 (ffff9627d1866100) 00010000:00010000:1.0:1788572539.865073:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.865075:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865081:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1617d7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865090:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865095:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865100:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865104:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.865107:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865112:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865119:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7e:0x0].0x0 (ffff9627d1866100) 00010000:00010000:1.0:1788572539.865121:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572539.865124:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.865129:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1617bb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.867096:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.867516:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382336640:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.867548:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.867884:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35888:x1875453360270720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 17208us (17333us total) trans 4294970177 rc 0/0 00000100:00100000:1.0:1788572539.867897:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3827 00000020:00080000:1.0:1788572539.867903:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:2.0:1788572539.870068:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572539.870275:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360270976 00000100:00100000:0.0:1788572539.870285:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360270976 00010000:00010000:0.0:1788572539.870329:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 15 pid: 6682 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.870422:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 911 00000100:00100000:1.0:1788572539.870435:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360270976:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.870451:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.870486:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 15 pid: 6682 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870500:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 15 pid: 6682 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870523:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb161760 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x40:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef40b expref: 15 pid: 6682 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870539:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.870544:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.870550:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870560:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870586:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.870623:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360270976:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 192us (3536us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.870631:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 911 00010000:00010000:1.0:1788572539.870674:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870684:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870708:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870717:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870725:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870733:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572539.870738:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870746:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870758:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.870766:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d8507034/0x85ace67feb1617d0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.870808:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360270336:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 15761us (31898us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.870815:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 976 00000100:00100000:1.0:1788572539.870872:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382336512:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.870895:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.877613:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.877668:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572539.877762:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360271360 00000100:00100000:0.0:1788572539.877773:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360271360 00000100:00100000:0.0:1788572539.877842:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 851 00000100:00100000:0.0:1788572539.877866:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35888:x1875453360271360:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572539.879419:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x73:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572539.879550:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970178 00010000:00010000:0.0:1788572539.879606:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.879623:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.879639:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.879648:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.879655:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.879661:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.879692:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572539.879786:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35888:x1875453360271360:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1919us (2179us total) trans 4294970179 rc 0/0 00000100:00100000:0.0:1788572539.879796:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 851 00000020:00080000:0.0:1788572539.879815:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572539.880175:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572539.880184:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.880190:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572539.880194:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.880202:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.880225:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x73:0x0].0x0 (ffff9627e7451100) 00010000:00010000:2.0:1788572539.880228:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572539.880233:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 1/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.880240:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb1617de lrc: 0/0,0 mode: --/PW res: [0x200000402:0x73:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.880258:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572539.880260:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572539.880262:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572539.880691:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360271616 00000100:00100000:3.0:1788572539.880700:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360271616 00000100:00100000:3.0:1788572539.880766:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3828 00000100:00100000:3.0:1788572539.880775:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35890:x1875453360271616:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.880852:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.880867:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.880877:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.881196:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: --/EX res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.881210:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.881363:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 4/0,1 mode: --/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.881409:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.881424:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 69 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.881441:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bced80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382337024:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.881543:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f14e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.885647:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360271104 00000100:00100000:1.0:1788572539.885663:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360271104 00010000:00010000:1.0:1788572539.885709:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 13 pid: 6681 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.885758:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 912 00000100:00100000:1.0:1788572539.885770:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e52450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360271104:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.885789:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.885819:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 13 pid: 6681 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.885834:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 13 pid: 6681 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.885862:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000795c09f0/0x85ace67feb161783 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x41:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef427 expref: 13 pid: 6681 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.885883:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.885887:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572539.885893:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.885905:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.885954:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.886046:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e52450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360271104:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 278us (15985us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.886054:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 912 00010000:00010000:1.0:1788572539.886117:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886127:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886168:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886178:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886186:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886193:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572539.886198:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886209:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886400:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.886411:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb1617c9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.886462:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360270208:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 31684us (47670us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.886485:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 975 00000100:00100000:1.0:1788572539.888169:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bced80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382337024:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572539.888199:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: --/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.890829:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.891842:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.892355:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360272000 00000100:00100000:3.0:1788572539.892367:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360272000 00000100:00100000:3.0:1788572539.892441:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3829 00000100:00100000:3.0:1788572539.892451:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35889:x1875453360272000:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.892554:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.892574:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.892589:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.893117:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: --/EX res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.893132:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.893147:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 4/0,1 mode: --/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.893327:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.893342:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 71 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.893360:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382337152:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572539.893502:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572539.894714:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360271744 00000100:00100000:0.0:1788572539.894729:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360271744 00010000:00010000:0.0:1788572539.894773:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 72 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.894818:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 913 00000100:00100000:0.0:1788572539.894827:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2350:x1875453360271744:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572539.894843:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572539.895080:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 72 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.895100:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 72 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.895122:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1616fe lrc: 0/0,0 mode: --/CR res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3d3 expref: 72 pid: 5839 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.895140:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:0.0:1788572539.895146:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: --/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.895153:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.895160:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.895194:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572539.895297:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.895305:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572539.895496:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970180 00000001:00080000:2.0:1788572539.895672:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970182 00000001:00080000:2.0:1788572539.895719:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970183 00000100:00100000:2.0:1788572539.895803:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382337280:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572539.895888:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2350:x1875453360271744:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1059us (5066us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572539.895896:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 913 00000100:00100000:3.0:1788572539.896680:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.896716:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.896772:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382337280 00000100:00100000:2.0:1788572539.896779:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382337280 00000100:00100000:2.0:1788572539.896828:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:2.0:1788572539.896837:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382337280:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572539.897055:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382337280:12345-0@lo:1000:mdt00_004.0 Request processed in 218us (376us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.897063:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:0.0:1788572539.897415:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967542 gen 2 00000004:00080000:0.0:1788572539.897421:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967535 committed transno 4294967542 00000004:00080000:0.0:1788572539.897439:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ff0a8e00, transno 4294967540 00000004:00080000:0.0:1788572539.897496:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ff0a9500, transno 4294967541 00000100:00100000:1.0:1788572539.901649:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.901694:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360272128 00000100:00100000:1.0:1788572539.901703:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360272128 00010000:00010000:1.0:1788572539.901746:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 74 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.901785:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 914 00000100:00100000:1.0:1788572539.901795:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360272128:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572539.901809:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572539.901816:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 74 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.901829:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 74 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.901864:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:1.0:1788572539.901871:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: --/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.901881:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.901894:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.901914:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572539.901972:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360272128:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 178us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.901980:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 914 00010000:00010000:2.0:1788572539.902950:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161752 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef3fd expref: 73 pid: 9040 timeout: 987 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572539.902974:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382337152:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572539.902995:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.903023:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.903030:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572539.903438:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970181 00000001:00080000:2.0:1788572539.903622:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970185 00000001:00080000:2.0:1788572539.903669:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970186 00000004:00080000:0.0:1788572539.904799:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ff0aa680, transno 4294967542 00000100:00100000:0.0:1788572539.904817:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382337280:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572539.905396:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382337408:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1788572539.905480:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572539.905556:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3fee40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572539.906141:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382337408 00000100:00100000:1.0:1788572539.906152:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382337408 00000100:00100000:1.0:1788572539.906353:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:1.0:1788572539.906364:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382337408:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572539.906457:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382337408:12345-0@lo:1000:mdt00_001.0 Request processed in 95us (977us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.906478:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1788572539.906596:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382337408:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1788572539.906932:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970187 00000100:00100000:3.0:1788572539.907155:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382337536:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572539.907454:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572539.907596:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51480 going to sleep for 11 seconds 00000001:00080000:1.0:1788572539.908043:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970184 00000100:00100000:1.0:1788572539.908102:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b5ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382337664:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572539.908133:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572539.908491:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582aa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572539.908565:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382337536 00000100:00100000:2.0:1788572539.908572:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382337536 00000100:00100000:3.0:1788572539.908609:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382337664 00000100:00100000:3.0:1788572539.908613:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382337664 00000100:00100000:3.0:1788572539.908684:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1788572539.908694:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382337664:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572539.908895:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:22536:x1875453382337664:12345-0@lo:1000:mdt00_004.0 Request processed in 203us (760us total) trans 4294967545 rc 0/0 00000100:00100000:3.0:1788572539.908904:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:1.0:1788572539.908977:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b5ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382337664:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572539.909044:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909054:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909062:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909073:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.909078:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909085:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909103:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:1.0:1788572539.909106:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572539.909112:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 2/0,0 mode: --/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909120:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1617ec lrc: 0/0,0 mode: --/EX res: [0x200000402:0x72:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909152:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909172:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909177:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909182:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.909184:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909190:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909201:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffff9627fb1e7300) 00010000:00010000:1.0:1788572539.909203:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572539.909207:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909212:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb15ecfe lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909244:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909257:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909267:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909274:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909279:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909285:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909289:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.909292:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909298:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909305:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffff9627fb1e7300) 00010000:00010000:1.0:1788572539.909308:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572539.909310:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909315:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161801 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909324:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909329:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909335:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909340:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572539.909343:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909348:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909356:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffff9627fb1e7300) 00010000:00010000:1.0:1788572539.909358:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572539.909361:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.909367:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1617e5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572539.909520:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35890:x1875453360271616:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 28743us (31849us total) trans 4294970189 rc 0/0 00000100:00100000:1.0:1788572539.909530:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3828 00000020:00080000:1.0:1788572539.909534:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572539.912927:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1788572539.913070:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382337536:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572539.913636:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5838:x1875453382337536:12345-0@lo:1000:mdt00_001.0 Request processed in 570us (6180us total) trans 4294967546 rc 0/0 00000100:00100000:0.0:1788572539.913711:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382337536:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1788572539.913765:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913775:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913782:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913787:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.913792:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913799:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913815:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627c2abc700) 00010000:00010000:0.0:1788572539.913818:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572539.913823:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 2/0,0 mode: --/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913830:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1617fa lrc: 0/0,0 mode: --/EX res: [0x200000402:0x74:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913861:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913867:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913873:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913883:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.913886:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913892:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913900:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572539.913902:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572539.913906:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913911:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb15ec79 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913934:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913960:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913969:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913976:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913981:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913986:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.913992:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.913995:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914001:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914042:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572539.914045:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572539.914048:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914054:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161808 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914065:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914071:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914077:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914083:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.914086:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914091:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914103:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572539.914105:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572539.914108:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.914114:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1617f3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.914288:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35889:x1875453360272000:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 21837us (22449us total) trans 4294970188 rc 0/0 00000100:00100000:0.0:1788572539.914298:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3829 00000020:00080000:0.0:1788572539.914303:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1788572539.914599:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572539.914604:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572539.914635:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 120 count 101 00000040:00080000:1.0:1788572539.914643:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:1.0:1788572539.914762:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59880 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382337792:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572539.914817:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582aa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572539.918334:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.918742:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360272384 00000100:00100000:1.0:1788572539.918754:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360272384 00000100:00100000:1.0:1788572539.918804:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 852 00000100:00100000:1.0:1788572539.918814:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e64c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35890:x1875453360272384:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572539.919075:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59880 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382337792:192.168.202.40@tcp:107:hsm_cdtr.0 00000004:00080000:1.0:1788572539.919393:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x72:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572539.919440:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970190 00010000:00010000:1.0:1788572539.919497:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.919515:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.919527:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.919537:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.919548:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.919554:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572539.919580:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572539.919661:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e64c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35890:x1875453360272384:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 847us (1335us total) trans 4294970191 rc 0/0 00000100:00100000:1.0:1788572539.919669:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 852 00000020:00080000:1.0:1788572539.919674:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572539.919835:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572539.919855:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.919862:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572539.919866:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.919874:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.919888:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x72:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:0.0:1788572539.919890:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572539.919894:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 1/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.919903:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16180f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x72:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.919920:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572539.919921:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572539.919923:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572539.920872:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.921113:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.921133:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.921154:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.921320:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360272768 00000100:00100000:1.0:1788572539.921328:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360272768 00000100:00100000:1.0:1788572539.921370:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360272896 00000100:00100000:1.0:1788572539.921374:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360272896 00000100:00100000:1.0:1788572539.921387:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360273024 00000100:00100000:1.0:1788572539.921389:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360273024 00000100:00100000:1.0:1788572539.921407:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3830 00000100:00100000:1.0:1788572539.921416:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35891:x1875453360272768:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572539.921445:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xc0:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572539.922095:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35891:x1875453360272768:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 678us (980us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.922104:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3830 00000100:00100000:1.0:1788572539.922124:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3831 00000100:00100000:1.0:1788572539.922133:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35892:x1875453360272896:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572539.922148:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xba:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572539.922396:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35892:x1875453360272896:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 264us (1260us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.922403:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3831 00000100:00100000:1.0:1788572539.922416:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3832 00000100:00100000:1.0:1788572539.922424:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35893:x1875453360273024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572539.922438:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbc:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572539.922575:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1788572539.922642:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35893:x1875453360273024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 219us (1485us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572539.922650:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3832 00000100:00100000:0.0:1788572539.924420:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360272640 00000100:00100000:0.0:1788572539.924436:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360272640 00000100:00100000:0.0:1788572539.924497:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 853 00000100:00100000:0.0:1788572539.924507:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35889:x1875453360272640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572539.924901:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x74:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572539.924946:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970192 00010000:00010000:0.0:1788572539.924989:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.925024:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.925039:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.925047:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.925053:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.925059:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.925085:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788572539.925152:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1788572539.925180:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35889:x1875453360272640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 674us (4312us total) trans 4294970193 rc 0/0 00000100:00100000:0.0:1788572539.925188:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 853 00000020:00080000:0.0:1788572539.925193:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572539.925563:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.925572:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572539.925577:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.925585:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.925598:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x74:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:3.0:1788572539.925601:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572539.925606:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.925613:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161816 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x74:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.925631:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572539.925633:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572539.925635:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572539.931877:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.931970:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.932659:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572539.932712:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360273152 00000100:00100000:0.0:1788572539.932722:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360273152 00000100:00100000:0.0:1788572539.932780:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360273280 00000100:00100000:0.0:1788572539.932783:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360273280 00000100:00100000:0.0:1788572539.932806:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3833 00000100:00100000:0.0:1788572539.932817:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35892:x1875453360273152:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00000100:00100000:3.0:1788572539.933087:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3834 00000100:00100000:3.0:1788572539.933097:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35891:x1875453360273280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.933176:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933196:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933211:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933233:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933241:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933248:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933272:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933279:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933285:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.933323:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc0:0x0]: returning size 3 00000004:00200000:3.0:1788572539.933327:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.933341:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933355:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933365:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933373:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933378:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.933382:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933390:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933402:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572539.933406:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.933411:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933417:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16181d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933431:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933437:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933442:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933447:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.933450:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933455:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933462:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x520072bf (ffff9627d599a200) 00010000:00010000:3.0:1788572539.933653:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.933657:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933664:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161824 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x520072bf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933679:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933685:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933691:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933696:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.933699:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933705:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933712:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:3.0:1788572539.933714:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.933717:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.933723:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16182b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.933796:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35891:x1875453360273280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 700us (1824us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.933804:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3834 00010000:00010000:0.0:1788572539.934442:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934458:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934483:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934499:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934506:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934513:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934536:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934545:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934552:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572539.934583:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xba:0x0]: returning size 3 00000004:00200000:0.0:1788572539.934588:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572539.934598:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934606:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934612:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934619:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934624:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.934627:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934634:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934644:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572539.934647:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572539.934664:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934675:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161832 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934688:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934693:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934699:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934704:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.934707:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934713:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934720:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x935a69ee (ffff9627e7451b00) 00010000:00010000:0.0:1788572539.934722:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572539.934725:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934730:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161839 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x935a69ee bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934745:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934751:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934757:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934762:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572539.934764:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934770:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934776:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572539.934779:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572539.934782:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.934787:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161840 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572539.934837:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35892:x1875453360273152:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 2019us (2974us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572539.935636:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3833 00000100:00100000:3.0:1788572539.936098:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360273408 00000100:00100000:3.0:1788572539.936107:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360273408 00000100:00100000:3.0:1788572539.936169:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3835 00000100:00100000:3.0:1788572539.936179:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35893:x1875453360273408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.936244:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936260:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936270:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936284:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936291:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936297:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936318:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936326:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936332:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.936362:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbc:0x0]: returning size 3 00000004:00200000:3.0:1788572539.936367:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.936378:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936388:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936394:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936400:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936405:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.936409:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936416:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936427:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572539.936429:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.936433:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936445:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161847 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936457:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936476:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936483:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936489:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.936491:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936498:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936505:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8cfc3e08 (ffff9627d599a800) 00010000:00010000:3.0:1788572539.936507:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.936510:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936515:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16184e lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8cfc3e08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936528:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936534:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936540:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936545:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.936548:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936554:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936561:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:3.0:1788572539.936563:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.936566:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.936572:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161855 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.936624:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35893:x1875453360273408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 446us (3966us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.936632:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3835 00000100:00100000:1.0:1788572539.945658:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.945901:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360273664 00000100:00100000:3.0:1788572539.945911:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360273664 00000100:00100000:3.0:1788572539.945984:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3836 00000100:00100000:3.0:1788572539.945995:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35892:x1875453360273664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.946057:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.946078:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16185c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef42e expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946191:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946206:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946232:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.946244:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xba:0x0]: returning size 3 00010000:00010000:3.0:1788572539.946342:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946351:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946358:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946364:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.946369:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946377:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946391:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572539.946394:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572539.946399:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946408:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161863 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946453:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16185c lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef42e expref: 82 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946473:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572539.946475:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572539.946480:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16185c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef42e expref: 82 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.946489:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:3.0:1788572539.946563:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35892:x1875453360273664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 569us (915us total) trans 4294970194 rc 301/301 00000100:00100000:3.0:1788572539.946571:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3836 00000100:00100000:1.0:1788572539.953729:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.953767:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.953837:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274176 00000100:00100000:3.0:1788572539.953849:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274176 00000100:00100000:3.0:1788572539.953903:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274304 00000100:00100000:3.0:1788572539.953907:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274304 00000100:00100000:3.0:1788572539.953931:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3837 00000100:00100000:3.0:1788572539.953949:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35893:x1875453360274176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.953973:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.953998:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161871 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef435 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954141:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954173:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954187:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.954198:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbc:0x0]: returning size 3 00010000:00010000:3.0:1788572539.954314:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954323:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954330:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954336:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.954340:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954348:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954361:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:3.0:1788572539.954364:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572539.954369:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954375:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161878 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954425:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161871 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef435 expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954434:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572539.954437:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572539.954440:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161871 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef435 expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.954449:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52c00, rc 0) 00000100:00100000:3.0:1788572539.954513:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35893:x1875453360274176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 567us (795us total) trans 4294970195 rc 301/301 00000100:00100000:3.0:1788572539.954521:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3837 00000100:00100000:3.0:1788572539.954540:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3838 00000100:00100000:3.0:1788572539.954548:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35892:x1875453360274304:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572539.954568:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xba:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572539.954634:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35892:x1875453360274304:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 85us (865us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.954640:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3838 00000100:00100000:0.0:1788572539.958579:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.958687:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274432 00000100:00100000:3.0:1788572539.958698:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274432 00000100:00100000:3.0:1788572539.958759:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3839 00000100:00100000:3.0:1788572539.958769:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35892:x1875453360274432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.958791:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.958812:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161886 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef443 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.958903:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.958918:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.958931:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.958956:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.958963:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.958970:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959484:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16189b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959495:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16189b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959503:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16189b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.959530:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:3.0:1788572539.959534:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.959552:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16189b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959568:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959575:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959581:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959586:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.959591:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959598:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959611:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572539.959614:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.959617:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959623:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16188d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959634:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959640:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959645:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959650:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.959653:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959658:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959665:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627d599a800) 00010000:00010000:3.0:1788572539.959667:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.959670:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959675:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161894 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959705:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161886 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef443 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959717:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16189b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef443 expref: 83 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.959725:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572539.959728:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572539.959730:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:3.0:1788572539.959788:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35892:x1875453360274432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1020us (1220us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.959796:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3839 00000100:00100000:1.0:1788572539.963881:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572539.963917:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.964031:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274560 00000100:00100000:2.0:1788572539.964041:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274560 00000100:00100000:2.0:1788572539.964090:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274048 00000100:00100000:2.0:1788572539.964093:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274048 00000100:00100000:2.0:1788572539.964118:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3840 00000100:00100000:2.0:1788572539.964127:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35893:x1875453360274560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572539.964171:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbc:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572539.964286:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35893:x1875453360274560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 159us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.964293:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3840 00000100:00100000:2.0:1788572539.964314:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3841 00000100:00100000:2.0:1788572539.964321:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35891:x1875453360274048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.964330:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.964351:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1618a2 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef43c expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964439:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964454:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964474:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572539.964486:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc0:0x0]: returning size 3 00010000:00010000:2.0:1788572539.964585:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964593:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964600:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964606:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.964611:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964619:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964630:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:2.0:1788572539.964633:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572539.964638:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964644:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618a9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964685:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1618a2 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef43c expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964695:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572539.964698:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572539.964702:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1618a2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef43c expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.964712:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572539.964743:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35891:x1875453360274048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 423us (826us total) trans 4294970196 rc 301/301 00000100:00100000:2.0:1788572539.964754:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3841 00000100:00100000:2.0:1788572539.972412:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572539.972518:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274816 00000100:00100000:0.0:1788572539.972529:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274816 00000100:00100000:0.0:1788572539.972592:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3842 00000100:00100000:0.0:1788572539.972603:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35892:x1875453360274816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572539.972626:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572539.972630:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572539.972659:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16189b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef443 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.972677:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572539.972683:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572539.972697:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1618b7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef44a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572539.973277:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x42:0x0], index 0: 0 00010000:00010000:0.0:1788572539.973481:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1618be lrc: 3/1,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.973495:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1618be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.973507:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1618be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.973567:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1618b7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef44a expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.973584:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1618be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef44a expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572539.973591:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451b00) 00010000:00010000:0.0:1788572539.973594:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572539.973596:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bace00, rc 0) 00000100:00100000:0.0:1788572539.973656:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35892:x1875453360274816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1055us (1253us total) trans 4294970197 rc 0/0 00000100:00100000:0.0:1788572539.973670:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3842 00000100:00100000:0.0:1788572539.977704:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.977773:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360274944 00000100:00100000:2.0:1788572539.977784:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360274944 00000100:00100000:2.0:1788572539.977848:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3843 00000100:00100000:2.0:1788572539.977859:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35891:x1875453360274944:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572539.977901:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xc0:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572539.978061:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35891:x1875453360274944:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 203us (365us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.978069:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3843 00000100:00100000:1.0:1788572539.982756:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.982947:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360275072 00000100:00100000:2.0:1788572539.982960:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360275072 00000100:00100000:2.0:1788572539.983954:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3844 00000100:00100000:2.0:1788572539.983971:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35893:x1875453360275072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.983994:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.984036:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1618cc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef451 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.984113:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.984129:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.984141:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.985350:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.985368:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.985377:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987447:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1618e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987482:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1618e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987495:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1618e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572539.987532:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:2.0:1788572539.987536:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572539.987558:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1618e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987573:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987580:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987587:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987597:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.987601:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987609:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987621:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572539.987624:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572539.987628:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987634:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1618d3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987647:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987652:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987658:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987662:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572539.987665:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987670:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987680:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627e7a95200) 00010000:00010000:2.0:1788572539.987683:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572539.987685:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987691:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1618da lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987735:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1618cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef451 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987750:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1618e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef451 expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.987757:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572539.987760:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572539.987762:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:2.0:1788572539.987844:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35893:x1875453360275072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3878us (5097us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572539.987853:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3844 00000100:00100000:1.0:1788572539.992926:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572539.993403:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572539.993803:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360275328 00000100:00100000:2.0:1788572539.993815:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360275328 00000100:00100000:2.0:1788572539.993959:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360275456 00000100:00100000:2.0:1788572539.993964:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360275456 00000100:00100000:2.0:1788572539.993995:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3845 00000100:00100000:2.0:1788572539.994022:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35893:x1875453360275328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572539.994048:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572539.994052:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572539.994078:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1618e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef451 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572539.994097:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572539.994103:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572539.994118:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1618e8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef458 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572539.994527:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3846 00000100:00100000:3.0:1788572539.994539:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35892:x1875453360275456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572539.994548:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572539.994559:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1618ef lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef45f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.994600:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.994614:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.994627:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.994641:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.994648:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.994654:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995034:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161904 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995046:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161904 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995053:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161904 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572539.995080:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:3.0:1788572539.995084:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572539.995101:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161904 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995111:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995119:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995126:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995132:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.995137:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995145:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995157:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572539.995160:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.995164:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995170:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1618f6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995180:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995190:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995196:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995202:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572539.995205:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995211:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995232:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627d599a800) 00010000:00010000:3.0:1788572539.995235:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572539.995237:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995243:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb1618fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995278:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1618ef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef45f expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995292:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161904 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef45f expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.995300:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572539.995302:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572539.995306:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:3.0:1788572539.995375:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35892:x1875453360275456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 839us (1973us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572539.995383:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3846 00000004:00080000:3.0:1788572539.998396:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x42:0x0], index 1: 0 00010000:00010000:3.0:1788572539.998617:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16190b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.998635:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16190b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.998647:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16190b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.998711:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1618e8 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef458 expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.998730:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16190b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef458 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572539.998740:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a200) 00010000:00010000:3.0:1788572539.998743:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572539.998746:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53000, rc 0) 00000100:00100000:3.0:1788572539.998874:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35893:x1875453360275328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4869us (5957us total) trans 4294970198 rc 0/0 00000100:00100000:3.0:1788572539.998884:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3845 00000100:00100000:3.0:1788572540.005662:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.005748:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360275712 00000100:00100000:1.0:1788572540.005759:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360275712 00000100:00100000:1.0:1788572540.005820:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3847 00000100:00100000:1.0:1788572540.005829:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35893:x1875453360275712:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.005901:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572540.005927:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb16190b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef458 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006315:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a200) 00010000:00010000:1.0:1788572540.006334:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161904 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef45f expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006343:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572540.006346:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572540.006369:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006383:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006392:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006407:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006414:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006420:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006637:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006650:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006657:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006663:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.006667:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006675:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006685:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572540.006688:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.006691:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006698:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161919 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006707:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006714:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006720:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006725:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.006729:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006736:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006743:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xa144963 (ffff9627d0ee2100) 00010000:00010000:1.0:1788572540.006745:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.006748:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.006754:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161920 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0xa144963 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.006854:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35893:x1875453360275712:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1017us (1193us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572540.006862:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3847 00000100:00100000:2.0:1788572540.010091:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.010187:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360275840 00000100:00100000:3.0:1788572540.010197:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360275840 00000100:00100000:3.0:1788572540.010260:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3848 00000100:00100000:3.0:1788572540.010271:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35893:x1875453360275840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.010291:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.010315:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161927 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef466 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.010352:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.010366:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.010378:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.010488:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161927 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.010505:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.010514:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572540.010517:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572540.010521:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:3.0:1788572540.010584:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35893:x1875453360275840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 314us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.010590:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3848 00000100:00100000:3.0:1788572540.011452:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.011524:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360276096 00000100:00100000:0.0:1788572540.011532:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360276096 00000100:00100000:0.0:1788572540.011581:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3849 00000100:00100000:0.0:1788572540.011590:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35891:x1875453360276096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.011604:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.011620:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161935 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef46d expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572540.012057:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x43:0x0], index 0: 0 00010000:00010000:0.0:1788572540.012524:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16193c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.012540:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16193c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.012550:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16193c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.012612:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161935 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef46d expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.012632:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16193c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef46d expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.012642:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451900) 00010000:00010000:0.0:1788572540.012645:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572540.012649:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2badc00, rc 0) 00000100:00100000:0.0:1788572540.012757:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35891:x1875453360276096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1167us (1306us total) trans 4294970199 rc 0/0 00000100:00100000:0.0:1788572540.012765:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3849 00000100:00100000:3.0:1788572540.016147:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.016364:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360276352 00000100:00100000:2.0:1788572540.016376:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360276352 00000100:00100000:2.0:1788572540.016445:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3850 00000100:00100000:2.0:1788572540.016456:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35893:x1875453360276352:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.016563:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.016580:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.016592:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572540.017935:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].52, 88 off12360 00000004:00010000:3.0:1788572540.018253:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ce67813c reply state 000000004c31256b transno 4294970200 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.018266:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572540.018449:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35893:x1875453360276352:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1991us (2310us total) trans 4294970200 rc 0/0 00000100:00100000:3.0:1788572540.018459:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3850 00010000:00010000:2.0:1788572540.022723:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.022741:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.022749:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.022755:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.022761:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.022769:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.022788:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:2.0:1788572540.022791:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572540.022796:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.022815:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16194a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.026140:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572540.026863:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.026899:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.026926:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.026964:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360276480 00000100:00100000:3.0:1788572540.026973:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360276480 00000100:00100000:3.0:1788572540.027041:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360276608 00000100:00100000:3.0:1788572540.027044:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360276608 00000100:00100000:3.0:1788572540.027060:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360276864 00000100:00100000:3.0:1788572540.027063:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360276864 00000100:00100000:3.0:1788572540.027083:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3851 00000100:00100000:3.0:1788572540.027092:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35893:x1875453360276480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.027114:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.027135:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef474 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.027212:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.027228:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.027252:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572540.027255:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572540.027259:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52c00, rc 0) 00000100:00100000:3.0:1788572540.027317:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35893:x1875453360276480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 226us (469us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.027324:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3851 00000100:00100000:2.0:1788572540.027432:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3852 00000100:00100000:2.0:1788572540.027443:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35891:x1875453360276608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.027456:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.027478:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161958 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef47b expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027534:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027545:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027555:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027575:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027582:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027588:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027910:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027920:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027928:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.027954:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:2.0:1788572540.027958:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.027975:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027987:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.027994:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028000:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028030:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.028035:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028041:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028052:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572540.028054:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.028058:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028064:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16195f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028076:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028081:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028087:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028092:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.028095:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028100:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028110:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627e7a95200) 00010000:00010000:2.0:1788572540.028112:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.028115:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028121:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161966 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028150:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161958 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028161:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 93 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028167:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572540.028169:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572540.028171:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04600, rc 0) 00000100:00100000:2.0:1788572540.028202:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35891:x1875453360276608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 763us (1302us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.028209:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3852 00000100:00100000:2.0:1788572540.028228:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3853 00000100:00100000:2.0:1788572540.028236:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35892:x1875453360276864:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.028268:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572540.028282:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1618be lrc: 0/0,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef44a expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028297:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451b00) 00010000:00010000:2.0:1788572540.028306:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572540.028316:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028326:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028334:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028366:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.028379:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 91 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.028392:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382337920:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.028451:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.035826:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.036243:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382337920:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.036273:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.036696:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360276992 00000100:00100000:3.0:1788572540.036704:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360276992 00010000:00010000:3.0:1788572540.036746:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 92 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.036790:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 915 00000100:00100000:3.0:1788572540.036800:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:2350:x1875453360276992:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.036828:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.036836:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 92 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.036858:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 92 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.036884:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16196d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef47b expref: 92 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.036904:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572540.036909:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.036921:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.036930:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.036959:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572540.037049:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:2350:x1875453360276992:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 251us (1233us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.037058:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 915 00010000:00010000:2.0:1788572540.037725:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.037735:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.037762:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.037773:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.037782:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038066:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038075:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038082:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038087:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.038092:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038099:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038109:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572540.038113:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.038116:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038123:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161974 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038134:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038139:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038145:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038150:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.038152:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038158:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038165:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x96452e4d (ffff9627e7a95a00) 00010000:00010000:2.0:1788572540.038168:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.038170:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.038175:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16197b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x96452e4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.038255:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35892:x1875453360276864:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 10017us (11324us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572540.038264:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3853 00000100:00100000:1.0:1788572540.039673:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.039763:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360277120 00000100:00100000:0.0:1788572540.039772:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360277120 00000100:00100000:0.0:1788572540.039837:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 977 00000100:00100000:0.0:1788572540.039856:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360277120:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.040331:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360277120:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 484us (664us total) trans 4294967547 rc 0/0 00000100:00100000:0.0:1788572540.040339:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 977 00000100:00080000:1.0:1788572540.040566:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788572540.045339:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.045509:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.045836:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.046192:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360277760 00000100:00100000:2.0:1788572540.046203:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360277760 00000100:00100000:2.0:1788572540.046272:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 854 00000100:00100000:2.0:1788572540.046283:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35893:x1875453360277760:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.046724:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35893:x1875453360277760:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 441us (886us total) trans 4294970201 rc 0/0 00000100:00100000:2.0:1788572540.046731:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 854 00000100:00100000:0.0:1788572540.047147:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360277248 00000100:00100000:0.0:1788572540.047154:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360277248 00000100:00100000:0.0:1788572540.047197:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360277504 00000100:00100000:0.0:1788572540.047201:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360277504 00000100:00100000:0.0:1788572540.047238:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3854 00000100:00100000:0.0:1788572540.047245:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35892:x1875453360277248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.047258:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.047281:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161982 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef482 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047315:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047328:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047343:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047422:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161982 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047439:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047447:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:0.0:1788572540.047450:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572540.047454:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:0.0:1788572540.047498:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35892:x1875453360277248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 253us (2171us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.047504:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3854 00000100:00100000:0.0:1788572540.047526:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3855 00000100:00100000:0.0:1788572540.047534:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35891:x1875453360277504:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.047564:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.047579:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16193c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef46d expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047591:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451900) 00010000:00010000:0.0:1788572540.047600:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.047616:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047629:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047635:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047647:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047654:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047661:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047840:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047860:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047867:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047872:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.047876:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047883:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047893:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572540.047897:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.047901:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047907:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161990 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047916:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047921:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047927:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047932:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.047934:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047941:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047947:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x1009c980 (ffff9627e7451c00) 00010000:00010000:0.0:1788572540.047950:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.047952:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.047958:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161997 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x1009c980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.048031:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35891:x1875453360277504:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 469us (2493us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572540.048039:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3855 00000100:00100000:3.0:1788572540.052925:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.052974:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.053035:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.053066:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360278016 00000100:00100000:3.0:1788572540.053077:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360278016 00000100:00100000:3.0:1788572540.053119:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360278272 00000100:00100000:3.0:1788572540.053122:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360278272 00000100:00100000:3.0:1788572540.053135:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360278400 00000100:00100000:3.0:1788572540.053138:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360278400 00000100:00100000:3.0:1788572540.053152:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3856 00000100:00100000:3.0:1788572540.053163:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35893:x1875453360278016:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.053315:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3857 00000100:00100000:1.0:1788572540.053324:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35892:x1875453360278272:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.053398:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.053414:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.053426:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572540.053840:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].54, 88 off12464 00000004:00010000:1.0:1788572540.053948:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000032ba95e0 reply state 00000000cb058c74 transno 4294970202 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.053958:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.054100:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3858 00000100:00100000:0.0:1788572540.054113:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35891:x1875453360278400:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.054127:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.054142:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1619a5 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef489 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.054169:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 3/1,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.054179:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.054187:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.054256:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1619a5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 97 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.054275:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 96 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.054283:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:0.0:1788572540.054286:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572540.054290:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad600, rc 0) 00000100:00100000:3.0:1788572540.055420:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35893:x1875453360278016:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 2249us (2499us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.055433:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3856 00000100:00100000:0.0:1788572540.055672:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35891:x1875453360278400:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1561us (2635us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.055683:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3858 00000100:00100000:1.0:1788572540.060459:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35892:x1875453360278272:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 7123us (7470us total) trans 4294970202 rc 0/0 00000100:00100000:1.0:1788572540.060483:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3857 00000100:00100000:3.0:1788572540.060651:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.060740:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.061311:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360278528 00000100:00100000:1.0:1788572540.061323:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360278528 00000100:00100000:1.0:1788572540.061378:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360278784 00000100:00100000:1.0:1788572540.061382:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360278784 00000100:00100000:0.0:1788572540.061561:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3859 00000100:00100000:0.0:1788572540.061574:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35893:x1875453360278528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.061597:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.061620:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb1619b3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef490 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.061698:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1619ba lrc: 3/1,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.061713:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1619ba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.061726:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1619ba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.061766:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.061785:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1619ba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.061813:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb1619b3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef490 expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.061831:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1619ba lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef490 expref: 96 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.061840:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:0.0:1788572540.061843:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572540.061847:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246c000, rc 0) 00000100:00100000:0.0:1788572540.061896:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35893:x1875453360278528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 325us (1246us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.061903:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3859 00000100:00100000:0.0:1788572540.061921:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3860 00000100:00100000:0.0:1788572540.061928:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35891:x1875453360278784:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.061993:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.062019:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.062028:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572540.062401:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].55, 88 off12552 00000004:00010000:0.0:1788572540.062509:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000067509d68 reply state 00000000d08ed2f7 transno 4294970203 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.062519:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.062570:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35891:x1875453360278784:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 641us (1828us total) trans 4294970203 rc 0/0 00000100:00100000:0.0:1788572540.062577:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3860 00000100:00100000:3.0:1788572540.067678:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.067708:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.067779:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.067808:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360279040 00000100:00100000:3.0:1788572540.067817:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360279040 00000100:00100000:3.0:1788572540.067865:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360279168 00000100:00100000:3.0:1788572540.067869:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360279168 00000100:00100000:3.0:1788572540.067892:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3861 00000100:00100000:3.0:1788572540.067901:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35892:x1875453360279040:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.067925:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.067953:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef49e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.068036:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.068052:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.068073:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:3.0:1788572540.068077:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572540.068081:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:3.0:1788572540.068131:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35892:x1875453360279040:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 231us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.068139:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3861 00000100:00100000:3.0:1788572540.068210:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3862 00010000:00010000:1.0:1788572540.068318:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.068326:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.068333:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.068339:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.068344:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.068351:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.068363:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:1.0:1788572540.068366:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572540.068371:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.068378:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619c1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.068422:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35891:x1875453360279168:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.068434:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.068443:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 2/0,0 mode: --/CR res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4a5 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.068481:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 98 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.068492:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 98 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.068505:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:3.0:1788572540.068507:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572540.068509:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:2.0:1788572540.068639:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360278912 00000100:00100000:2.0:1788572540.068644:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360278912 00000100:00100000:2.0:1788572540.068679:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 978 00000100:00100000:2.0:1788572540.068686:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360278912:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.068695:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.068709:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef497 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572540.068733:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x42:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737b, blocks: 0x1 00010000:00010000:2.0:1788572540.068747:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=888s v=(0 0) 00010000:00010000:2.0:1788572540.068751:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.068761:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.068776:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.068786:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05400, rc 0) 00000100:00100000:3.0:1788572540.068826:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35891:x1875453360279168:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 609us (1046us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.068834:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3862 00010000:00010000:0.0:1788572540.069508:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.069518:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.069525:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.069531:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.069535:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.069542:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.069554:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:0.0:1788572540.069557:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572540.069561:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.069568:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16199e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.071773:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360278912:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3073us (4091us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.071787:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 978 00000100:00100000:3.0:1788572540.072635:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.072812:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360279296 00000100:00100000:0.0:1788572540.072822:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360279296 00000100:00100000:0.0:1788572540.072886:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 979 00000100:00100000:0.0:1788572540.072896:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360279296:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.073203:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360279296:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 307us (569us total) trans 4294967546 rc 0/0 00000100:00100000:0.0:1788572540.073210:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 979 00000100:00080000:2.0:1788572540.073333:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572540.078087:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.078125:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.078209:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360279680 00000100:00100000:2.0:1788572540.078230:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360279680 00000100:00100000:2.0:1788572540.078287:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 980 00000100:00100000:2.0:1788572540.078298:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360279680:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.078612:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360279680:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 315us (485us total) trans 4294967547 rc 0/0 00000100:00100000:2.0:1788572540.078621:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 980 00000100:00100000:2.0:1788572540.078648:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360279552 00000100:00100000:2.0:1788572540.078652:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360279552 00000100:00100000:2.0:1788572540.078682:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3863 00000100:00100000:2.0:1788572540.078689:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35893:x1875453360279552:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.078747:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35893:x1875453360279552:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 58us (671us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.078754:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3863 00000100:00100000:3.0:1788572540.083130:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.083247:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360279808 00000100:00100000:2.0:1788572540.083258:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360279808 00000100:00100000:2.0:1788572540.083311:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 981 00000100:00100000:2.0:1788572540.083321:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35893:x1875453360279808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.083338:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.083355:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 2/0,0 mode: --/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4ac expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.083390:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.083407:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.083411:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.083417:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 4/0,0 mode: --/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.083454:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.083479:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.083494:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382338048:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.083574:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.087100:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.087236:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360280064 00000100:00100000:3.0:1788572540.087247:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360280064 00000100:00100000:3.0:1788572540.087308:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 855 00000100:00100000:3.0:1788572540.087320:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35891:x1875453360280064:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.087737:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35891:x1875453360280064:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 418us (645us total) trans 4294970204 rc 0/0 00000100:00100000:3.0:1788572540.087750:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 855 00000100:00100000:2.0:1788572540.087961:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382338048:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.087992:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 3/0,0 mode: --/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.088051:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572540.088074:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35893:x1875453360279808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4756us (4954us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.088082:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 981 00000100:00100000:1.0:1788572540.090270:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.091042:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360280192 00000100:00100000:3.0:1788572540.091052:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360280192 00010000:00010000:3.0:1788572540.091093:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.091135:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 916 00000100:00100000:3.0:1788572540.091144:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360280192:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.091162:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.091184:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.091199:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.091231:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000074d56732/0x85ace67feb1619d6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef497 expref: 14 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.091248:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.091252:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.091261:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.091272:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.091318:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.091335:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b52d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382338176:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572540.091423:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3fe0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.092919:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.093105:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360280448 00000100:00100000:3.0:1788572540.093114:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360280448 00000100:00100000:3.0:1788572540.093225:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3864 00000100:00100000:3.0:1788572540.093236:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35891:x1875453360280448:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.093350:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35891:x1875453360280448:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 116us (440us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.093359:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3864 00000100:00100000:2.0:1788572540.100261:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.100389:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.100607:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b52d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382338176:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572540.100632:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.100693:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360280192:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 9549us (10431us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.100706:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 916 00000100:00100000:0.0:1788572540.100749:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360280960 00000100:00100000:0.0:1788572540.100756:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360280960 00000100:00100000:0.0:1788572540.100798:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3865 00000100:00100000:0.0:1788572540.100807:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35891:x1875453360280960:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.100827:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.100847:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1619e4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4b3 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.100922:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb1619eb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.100950:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb1619eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.100961:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb1619eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.100997:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.101040:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb1619eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.101066:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1619e4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4b3 expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.101083:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb1619eb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4b3 expref: 99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.101092:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:0.0:1788572540.101095:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572540.101099:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031600, rc 0) 00000100:00100000:0.0:1788572540.101123:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35891:x1875453360280960:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 317us (733us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.101130:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3865 00000100:00100000:1.0:1788572540.101700:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360280704 00000100:00100000:1.0:1788572540.101708:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360280704 00000100:00100000:1.0:1788572540.101754:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 856 00000100:00100000:1.0:1788572540.101763:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35892:x1875453360280704:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.102189:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35892:x1875453360280704:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 425us (1938us total) trans 4294970205 rc 0/0 00000100:00100000:1.0:1788572540.102197:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 856 00000100:00100000:2.0:1788572540.103747:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.103825:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360280832 00000100:00100000:1.0:1788572540.103833:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360280832 00002000:00100000:1.0:1788572540.103885:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x42:0x0] (0->2) req@ffff9627fb470e00 x1875453360280832/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/0 e 0 to 0 dl 1788572551 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572540.103897:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb470e00 with x1875453360280832 ext(0->2) 00002000:00010000:1.0:1788572540.103903:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb470e00 00000100:00100000:1.0:1788572540.103931:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 226 00000100:00100000:1.0:1788572540.104186:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb470e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360280832:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.104625:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb470e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360280832:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 685us (882us total) trans 4294967548 rc 0/0 00000100:00100000:1.0:1788572540.104634:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 226 00002000:00100000:1.0:1788572540.104641:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x42:0x0] (0->2) req@ffff9627fb470e00 x1875453360280832/t4294967548(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/448 e 0 to 0 dl 1788572551 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572540.104655:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb470e00 with x1875453360280832 ext(0->2) 00002000:00010000:1.0:1788572540.104658:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb470e00 00000100:00100000:2.0:1788572540.109998:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.110072:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.110102:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.110169:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360281216 00000100:00100000:1.0:1788572540.110179:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360281216 00000100:00100000:1.0:1788572540.110246:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360281344 00000100:00100000:1.0:1788572540.110250:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360281344 00000100:00100000:1.0:1788572540.110274:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3866 00000100:00100000:1.0:1788572540.110284:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35892:x1875453360281216:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.110406:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35892:x1875453360281216:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 122us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.110413:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3866 00000100:00100000:1.0:1788572540.110432:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3867 00000100:00100000:1.0:1788572540.110439:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35893:x1875453360281344:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.110583:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360281472 00000100:00100000:0.0:1788572540.110588:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360281472 00000100:00100000:0.0:1788572540.110620:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 982 00000100:00100000:0.0:1788572540.110627:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360281472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.110638:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.110658:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4ba expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572540.110688:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x43:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737b, blocks: 0x1 00010000:00010000:0.0:1788572540.110706:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=888s v=(0 0) 00010000:00010000:0.0:1788572540.110711:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572540.110721:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.110740:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.110754:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031c00, rc 0) 00000100:00100000:0.0:1788572540.110776:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360281472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 149us (673us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.110783:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 982 00000100:00100000:0.0:1788572540.131523:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.131617:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.131648:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360281600 00000100:00100000:0.0:1788572540.131658:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360281600 00000100:00100000:0.0:1788572540.131716:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360281856 00000100:00100000:0.0:1788572540.131719:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360281856 00000100:00100000:0.0:1788572540.131744:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3868 00000100:00100000:0.0:1788572540.131754:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35892:x1875453360281600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.131774:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.131796:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619f9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4c1 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.131888:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161a00 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.131903:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161a00 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.131915:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161a00 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.131957:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.131976:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161a00 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.132029:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1619f9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c1 expref: 104 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.132047:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161a00 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c1 expref: 103 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.132062:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:0.0:1788572540.132064:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572540.132068:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246d800, rc 0) 00000100:00100000:0.0:1788572540.132125:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35892:x1875453360281600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 372us (612us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.132132:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3868 00000100:00100000:0.0:1788572540.132152:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3869 00000100:00100000:0.0:1788572540.132159:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35891:x1875453360281856:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.132203:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35891:x1875453360281856:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 45us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.132210:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3869 00000100:00100000:2.0:1788572540.141049:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.141128:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.141739:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360281984 00000100:00100000:3.0:1788572540.141750:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360281984 00000100:00100000:3.0:1788572540.141803:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360282112 00000100:00100000:3.0:1788572540.141806:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360282112 00000100:00100000:3.0:1788572540.141829:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 983 00000100:00100000:3.0:1788572540.141838:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360281984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.141855:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.141876:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4c8 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572540.141907:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x42:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737b, blocks: 0x1 00010000:00010000:3.0:1788572540.141926:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=888s v=(0 0) 00010000:00010000:3.0:1788572540.141932:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.141954:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 15 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.141971:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 15 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.141985:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:3.0:1788572540.142067:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360281984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 229us (1062us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.142073:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 983 00000100:00100000:3.0:1788572540.142090:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 984 00000100:00100000:3.0:1788572540.142097:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35891:x1875453360282112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.142104:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.142113:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4cf expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.142131:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 15 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.142139:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.142142:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.142146:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: --/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.142177:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 15 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.142191:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 15 pid: 6681 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.142206:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b53480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382338304:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572540.143585:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3fee40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.148195:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.148237:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360282240 00000100:00100000:1.0:1788572540.148246:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360282240 00010000:00010000:1.0:1788572540.148306:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 16 pid: 6681 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.148353:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 917 00000100:00100000:1.0:1788572540.148364:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360282240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.148380:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.148405:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 16 pid: 6681 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.148426:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 16 pid: 6681 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.148452:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.148457:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.148477:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 16 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.148488:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 16 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.148527:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 16 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.148543:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4920e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382338432:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572540.148616:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194480 going to sleep for 11 seconds 00010000:00010000:0.0:1788572540.149254:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb1619f2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ba expref: 16 pid: 6681 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.149273:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b53480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382338304:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572540.149296:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.149309:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:0.0:1788572540.149350:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35891:x1875453360282112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 7252us (8220us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.149357:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 984 00000100:00100000:0.0:1788572540.153414:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.153502:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4920e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382338432:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572540.153530:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.153580:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360282240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5217us (5396us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.153590:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 917 00000100:00100000:0.0:1788572540.153698:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.154485:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360282624 00000100:00100000:1.0:1788572540.154495:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360282624 00002000:00100000:1.0:1788572540.154544:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x43:0x0] (0->2) req@ffff9626c13bfb80 x1875453360282624/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/0 e 0 to 0 dl 1788572551 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572540.154556:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bfb80 with x1875453360282624 ext(0->2) 00002000:00010000:1.0:1788572540.154562:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c13bfb80 00000100:00100000:1.0:1788572540.154595:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 227 00000100:00100000:1.0:1788572540.154606:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360282624:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.155140:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360282624:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 534us (1440us total) trans 4294967548 rc 0/0 00000100:00100000:1.0:1788572540.155149:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 227 00002000:00100000:1.0:1788572540.155157:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x43:0x0] (0->2) req@ffff9626c13bfb80 x1875453360282624/t4294967548(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/448 e 0 to 0 dl 1788572551 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572540.155166:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bfb80 with x1875453360282624 ext(0->2) 00002000:00010000:1.0:1788572540.155169:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c13bfb80 00000100:00100000:1.0:1788572540.155195:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360282496 00000100:00100000:1.0:1788572540.155199:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360282496 00000100:00100000:1.0:1788572540.155841:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3870 00000100:00100000:1.0:1788572540.155852:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35892:x1875453360282496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.156302:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35892:x1875453360282496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 451us (2896us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.156311:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3870 00000100:00100000:2.0:1788572540.165589:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.166280:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360282752 00000100:00100000:2.0:1788572540.166291:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360282752 00000100:00100000:2.0:1788572540.166344:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3871 00000100:00100000:2.0:1788572540.166354:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35891:x1875453360282752:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.166680:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35891:x1875453360282752:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 327us (1100us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.166689:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3871 00000100:00100000:1.0:1788572540.172827:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.172910:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.174858:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360282880 00000100:00100000:2.0:1788572540.174876:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360282880 00000100:00100000:2.0:1788572540.175170:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283008 00000100:00100000:2.0:1788572540.175176:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283008 00000100:00100000:2.0:1788572540.175208:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 985 00000100:00100000:2.0:1788572540.175230:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35892:x1875453360282880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.175250:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.175270:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4d6 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.175306:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.175322:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.175326:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.175331:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 18 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.175370:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.175385:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.175402:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382338560:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.175495:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.176113:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 986 00000100:00100000:0.0:1788572540.176134:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360283008:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.189689:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382338560:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572540.189724:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 18 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.189747:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:0.0:1788572540.189803:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:35892:x1875453360282880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 14587us (16985us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.189812:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 985 00000100:00100000:0.0:1788572540.194924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.194969:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283136 00000100:00100000:0.0:1788572540.194978:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283136 00010000:00010000:0.0:1788572540.195047:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.195085:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 918 00000100:00100000:0.0:1788572540.195096:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360283136:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572540.195112:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.195139:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.195154:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.195178:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161a07 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c8 expref: 18 pid: 6679 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.195196:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572540.195200:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572540.195210:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.195221:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.195262:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.195277:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382338688:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572540.195355:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3fe0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.202102:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.202130:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283264 00000100:00100000:1.0:1788572540.202137:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283264 00002000:00100000:1.0:1788572540.202187:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x42:0x0] (0->2) req@ffff9626c4922d80 x1875453360283264/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/0 e 0 to 0 dl 1788572551 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572540.202198:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4922d80 with x1875453360283264 ext(0->2) 00002000:00010000:1.0:1788572540.202203:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4922d80 00000100:00100000:0.0:1788572540.202355:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382338688:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572540.202379:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.202431:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360283136:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7337us (7517us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.202445:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 918 00000100:00100000:1.0:1788572540.205631:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 228 00000100:00100000:1.0:1788572540.205654:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360283264:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.206106:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360283264:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 459us (4013us total) trans 4294967549 rc 0/0 00000100:00100000:1.0:1788572540.206116:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 228 00002000:00100000:1.0:1788572540.206125:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x42:0x0] (0->2) req@ffff9626c4922d80 x1875453360283264/t4294967549(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/448 e 0 to 0 dl 1788572551 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572540.206136:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4922d80 with x1875453360283264 ext(0->2) 00002000:00010000:1.0:1788572540.206142:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4922d80 00000100:00100000:3.0:1788572540.213418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.213500:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283392 00000100:00100000:1.0:1788572540.213511:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283392 00000100:00100000:1.0:1788572540.213570:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3872 00000100:00100000:1.0:1788572540.213580:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35892:x1875453360283392:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.213710:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35892:x1875453360283392:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 130us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.213719:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3872 00000100:00100000:1.0:1788572540.217462:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35893:x1875453360281344:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 107010us (107377us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.217486:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3867 00000001:00080000:2.0:1788572540.217588:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970203 is committed 00000020:00080000:2.0:1788572540.217616:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 93 batchid updated 00000001:00080000:2.0:1788572540.217637:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970205 is committed 00000020:00080000:2.0:1788572540.217654:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c63cc780 refcount 2 committed 0 result 0 batchid 96 00000020:00080000:2.0:1788572540.217659:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627e77349c0 00000020:00080000:2.0:1788572540.217663:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aea00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c375f000 00000020:00080000:2.0:1788572540.217671:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c5102900 refcount 2 committed 0 result 0 batchid 94 00000020:00080000:2.0:1788572540.217674:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae200 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c5102180 00000020:00080000:2.0:1788572540.217678:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffeb5c00 00000020:00080000:2.0:1788572540.217717:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c3194240 refcount 2 committed 0 result 0 batchid 95 00000020:00080000:2.0:1788572540.217721:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31ded00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c3194cc0 00000020:00080000:2.0:1788572540.217723:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31def00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c375d400 00000020:00080000:1.0:1788572540.217781:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 96 committed batchid 93 00000020:00080000:1.0:1788572540.217976:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 93 00000100:00100000:3.0:1788572540.224607:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.224649:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.224977:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283520 00000100:00100000:2.0:1788572540.224988:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283520 00000100:00100000:2.0:1788572540.225065:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283648 00000100:00100000:2.0:1788572540.225068:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283648 00000100:00100000:2.0:1788572540.225094:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 987 00000100:00100000:2.0:1788572540.225102:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360283520:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.226051:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 988 00000100:00100000:3.0:1788572540.226066:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360283648:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.233818:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360283008:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 57683us (60895us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.233834:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 986 00000001:00080000:0.0:1788572540.233875:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967546 is committed 00000001:00080000:0.0:1788572540.233885:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967548 is committed 00000001:00080000:0.0:1788572540.233901:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967545 is committed 00000100:00100000:2.0:1788572540.243249:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.243389:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360283904 00000100:00100000:1.0:1788572540.243400:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360283904 00000100:00100000:1.0:1788572540.243458:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3873 00000100:00100000:1.0:1788572540.243478:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35891:x1875453360283904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.243540:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.243567:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb1619eb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4b3 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.243587:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:1.0:1788572540.243592:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572540.243622:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.243636:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.243645:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572540.243808:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d53b8d88 reply state 0000000043f9e59f transno 4294970206 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.243818:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572540.243925:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35891:x1875453360283904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 448us (684us total) trans 4294970206 rc 0/0 00000100:00100000:1.0:1788572540.243933:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3873 00010000:00010000:1.0:1788572540.247199:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.247214:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.247231:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.247238:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.247243:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.247252:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.247268:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:1.0:1788572540.247272:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.247278:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.247285:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a1c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.253672:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.254327:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360284032 00000100:00100000:3.0:1788572540.254337:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360284032 00000100:00100000:3.0:1788572540.254392:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 989 00000100:00100000:3.0:1788572540.254401:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360284032:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572540.254633:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x43:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572540.254636:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x43:0x0] updating lvb mtime from disk: 0 -> 1788572515 00002000:00010000:3.0:1788572540.254639:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x43:0x0] updating lvb atime from disk: 0 -> 1788572515 00002000:00010000:3.0:1788572540.254641:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x43:0x0] updating lvb ctime from disk: 1788572539 -> 1788572527 00002000:00010000:3.0:1788572540.254644:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x43:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572540.254696:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360284032:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 295us (1033us total) trans 4294967550 rc 0/0 00000100:00100000:3.0:1788572540.254702:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 989 00000100:00100000:1.0:1788572540.257085:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.258047:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360284288 00000100:00100000:3.0:1788572540.258058:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360284288 00000100:00100000:3.0:1788572540.258116:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3874 00000100:00100000:3.0:1788572540.258126:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35891:x1875453360284288:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.258198:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.258228:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.258239:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.258392:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.258409:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 65 pid: 5837 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.258425:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382338816:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572540.258517:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.269343:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.269531:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360284416 00000100:00100000:3.0:1788572540.269541:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360284416 00010000:00010000:3.0:1788572540.269583:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 66 pid: 5837 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.269628:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 919 00000100:00100000:3.0:1788572540.269638:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e52050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360284416:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.269653:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.269660:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 66 pid: 5837 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.269674:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 66 pid: 5837 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.269698:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:3.0:1788572540.269703:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.269714:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.269723:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.269743:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572540.269800:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e52050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360284416:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 163us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.269808:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 919 00010000:00010000:1.0:1788572540.273765:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1619ac lrc: 0/0,0 mode: --/PR res: [0x200000402:0x77:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef489 expref: 65 pid: 5837 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.273789:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382338816:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.273812:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.273960:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.273969:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274029:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274043:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274052:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274058:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.274062:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274071:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274088:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:1.0:1788572540.274092:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572540.274096:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.274101:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a23 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.274189:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35891:x1875453360284288:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 16064us (17113us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572540.274198:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3874 00000100:00100000:0.0:1788572540.288909:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.292000:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360284544 00000100:00100000:2.0:1788572540.293663:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360284544 00000100:00100000:2.0:1788572540.293793:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 990 00000100:00100000:2.0:1788572540.293805:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360284544:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.293854:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.293877:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.293886:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.293890:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572540.293896:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 4/1,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.293937:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.293961:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 15 pid: 6679 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.293977:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382338944:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.294099:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.295525:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360283520:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 70413us (70927us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.295538:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 987 00000001:00080000:0.0:1788572540.295575:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967549 is committed 00000100:00100000:1.0:1788572540.298691:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.299055:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382338944:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.299085:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.302072:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.302436:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360284928 00000100:00100000:0.0:1788572540.302446:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360284928 00000100:00100000:0.0:1788572540.302517:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3875 00000100:00100000:0.0:1788572540.302528:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35892:x1875453360284928:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.302586:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.302614:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161a00 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4c1 expref: 64 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.302634:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:0.0:1788572540.302639:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.302674:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.302689:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.302699:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572540.302861:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000213a2ddf reply state 00000000680a8921 transno 4294970207 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.302870:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.302980:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35892:x1875453360284928:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 453us (917us total) trans 4294970207 rc 0/0 00000100:00100000:0.0:1788572540.302987:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3875 00000100:00100000:0.0:1788572540.307409:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572540.307675:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.307688:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.307698:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.307704:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.307709:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.307718:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.307739:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:1.0:1788572540.307742:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.307748:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.307755:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161a31 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.308544:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360284672 00000100:00100000:2.0:1788572540.308555:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360284672 00010000:00010000:2.0:1788572540.308600:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 13 pid: 6679 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.308676:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 920 00000100:00100000:2.0:1788572540.308686:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360284672:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572540.308702:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572540.308727:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 13 pid: 6679 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.308743:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 13 pid: 6679 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.308766:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161a0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4cf expref: 13 pid: 6679 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.308782:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572540.308786:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572540.308794:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.308804:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.308840:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572540.308908:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360284672:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 223us (10227us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.308916:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 920 00010000:00010000:1.0:1788572540.308975:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.308985:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.309032:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.309041:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572540.309045:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967547 is committed 00010000:00010000:1.0:1788572540.309057:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.309066:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc14d24f0) 00000001:00080000:3.0:1788572540.309067:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967548 is committed 00010000:00010000:1.0:1788572540.309072:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572540.309080:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967546 is committed 00010000:00010000:1.0:1788572540.309083:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.309098:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.309107:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ed1c992/0x85ace67feb161a2a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.309161:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360283648:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 83099us (84510us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.309175:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 988 00000100:00100000:3.0:1788572540.309190:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360285056 00000100:00100000:3.0:1788572540.309195:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360285056 00000100:00100000:3.0:1788572540.309220:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 991 00000100:00100000:3.0:1788572540.309228:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360285056:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.309516:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360284544:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 15713us (20617us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.309525:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 990 00002000:00010000:3.0:1788572540.309568:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x42:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572540.309573:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x42:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:3.0:1788572540.309576:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x42:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:3.0:1788572540.309579:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x42:0x0] updating lvb ctime from disk: 1788572539 -> 1788572527 00002000:00010000:3.0:1788572540.309582:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x42:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572540.309612:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360285056:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 383us (2216us total) trans 4294967551 rc 0/0 00000100:00100000:3.0:1788572540.309619:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 991 00000100:00100000:1.0:1788572540.314810:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.314860:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.314863:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360285312 00000100:00100000:3.0:1788572540.314876:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360285312 00000100:00100000:3.0:1788572540.314934:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360285568 00000100:00100000:3.0:1788572540.314937:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360285568 00000100:00100000:3.0:1788572540.314957:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3876 00000100:00100000:3.0:1788572540.314966:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35892:x1875453360285312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.315056:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315075:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315085:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315120:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315133:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 65 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.315147:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239b800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382339072:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572540.315231:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.315253:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3877 00000100:00100000:3.0:1788572540.315263:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35893:x1875453360285568:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.315298:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.315322:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1619ba lrc: 0/0,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef490 expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315352:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572540.315358:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.315375:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572540.315384:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315404:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315415:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572540.315603:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006db2d3ba reply state 000000009271a1f8 transno 4294970208 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.315615:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572540.315702:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35893:x1875453360285568:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 440us (841us total) trans 4294970208 rc 0/0 00000100:00100000:3.0:1788572540.315709:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3877 00000100:00100000:3.0:1788572540.315726:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360285824 00000100:00100000:3.0:1788572540.315729:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360285824 00000100:00100000:3.0:1788572540.315756:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3878 00000100:00100000:3.0:1788572540.315762:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35891:x1875453360285824:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.315799:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315809:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315817:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315967:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315976:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.315982:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 4/0,1 mode: --/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.316021:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.316036:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.316047:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382339200:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572540.316083:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5ef00 going to sleep for 11 seconds 00010000:00010000:1.0:1788572540.319732:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.319755:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.319763:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.319769:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.319774:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.319783:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.319799:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:1.0:1788572540.319803:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.319807:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.319814:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161a3f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.320704:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.320790:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360285952 00000100:00100000:0.0:1788572540.320802:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360285952 00000100:00100000:0.0:1788572540.320866:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 992 00000100:00100000:0.0:1788572540.320876:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360285952:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572540.321175:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x42:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572540.321179:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x42:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:0.0:1788572540.321181:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x42:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:0.0:1788572540.321183:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x42:0x0] updating lvb ctime from disk: 1788572539 -> 1788572527 00002000:00010000:0.0:1788572540.321186:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x42:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572540.321261:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360285952:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 386us (563us total) trans 4294967549 rc 0/0 00000100:00100000:0.0:1788572540.321270:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 992 00000100:00100000:3.0:1788572540.325584:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239b800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382339072:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.325616:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.325638:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382339200:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.325655:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.326556:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.326679:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360286208 00000100:00100000:0.0:1788572540.326687:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360286208 00010000:00010000:0.0:1788572540.326728:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.326775:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 921 00000100:00100000:0.0:1788572540.326786:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360286208:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572540.326801:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.326809:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.326819:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.326841:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1619cf lrc: 0/0,0 mode: --/CR res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4a5 expref: 67 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.326856:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:0.0:1788572540.326861:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.326871:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.326879:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.326911:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.326979:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360286208:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 194us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.326988:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 921 00010000:00010000:1.0:1788572540.327199:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.327221:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572540.327428:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970209 00000001:00080000:1.0:1788572540.327607:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970210 00000001:00080000:1.0:1788572540.327655:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970211 00000100:00100000:1.0:1788572540.327743:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382339328:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572540.327797:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572540.327836:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3251480 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.329724:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382339328 00000100:00100000:3.0:1788572540.329738:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382339328 00000100:00100000:3.0:1788572540.329813:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:3.0:1788572540.329823:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382339328:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572540.329934:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382339328:12345-0@lo:1000:mdt00_004.0 Request processed in 113us (2136us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.329953:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00080000:2.0:1788572540.330100:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572540.330538:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967545 gen 2 00000004:00080000:1.0:1788572540.330545:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967545 committed transno 4294967545 00000100:00100000:1.0:1788572540.330568:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382339328:0@lo:1000:mdt00_004.0 00000001:00080000:1.0:1788572540.330963:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970212 00000100:00100000:1.0:1788572540.331082:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382339456:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572540.331129:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572540.331224:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a540 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.332633:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382339456 00000100:00100000:3.0:1788572540.332646:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382339456 00000100:00100000:3.0:1788572540.332713:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:3.0:1788572540.332723:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382339456:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572540.333512:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382339456:12345-0@lo:1000:mdt00_004.0 Request processed in 788us (2380us total) trans 4294967552 rc 0/0 00000100:00100000:3.0:1788572540.333523:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1788572540.333617:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382339456:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572540.333664:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333676:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333685:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333691:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.333695:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333703:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333720:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627e7451d00) 00010000:00010000:1.0:1788572540.333722:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572540.333727:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333733:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161a4d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x77:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333769:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333776:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333781:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333785:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.333788:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333794:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333802:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:1.0:1788572540.333804:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572540.333807:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333816:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92400/0x85ace67feb15ed21 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333837:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333915:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333926:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.333998:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334112:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334118:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334124:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.334127:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334135:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334143:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:1.0:1788572540.334146:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.334148:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334157:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161a54 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334168:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334174:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334181:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334187:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.334189:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334195:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334202:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:1.0:1788572540.334204:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572540.334208:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.334289:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161a46 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.334622:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35891:x1875453360285824:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 18856us (19244us total) trans 4294970213 rc 0/0 00000100:00100000:1.0:1788572540.334633:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3878 00000020:00080000:1.0:1788572540.334638:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572540.341188:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.341238:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.341261:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.341326:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360286720 00000100:00100000:1.0:1788572540.341337:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360286720 00000100:00100000:1.0:1788572540.341400:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 857 00000100:00100000:1.0:1788572540.341411:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35891:x1875453360286720:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572540.342123:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x77:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572540.342233:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970214 00010000:00010000:1.0:1788572540.342284:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342301:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342312:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342321:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342327:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342333:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342360:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572540.342449:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35891:x1875453360286720:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1038us (1185us total) trans 4294970215 rc 0/0 00000100:00100000:1.0:1788572540.342457:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 857 00000020:00080000:1.0:1788572540.342474:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788572540.342502:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360286080 00000100:00100000:1.0:1788572540.342508:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360286080 00010000:00010000:1.0:1788572540.342528:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 71 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.342552:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 922 00000100:00100000:1.0:1788572540.342559:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2349:x1875453360286080:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.342570:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.342576:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 71 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342588:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 71 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342606:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161989 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x75:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef482 expref: 71 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.342998:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:1.0:1788572540.343027:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.343037:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.343044:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.343070:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.343093:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360286080:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 535us (1915us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.343104:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 922 00000100:00100000:2.0:1788572540.343505:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360286464 00000100:00100000:2.0:1788572540.343511:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360286464 00000100:00100000:2.0:1788572540.343553:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3879 00000100:00100000:2.0:1788572540.343561:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35893:x1875453360286464:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.343607:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.343620:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.343628:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.343663:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.343677:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.343699:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f90a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382339584:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.343756:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5000 going to sleep for 11 seconds 00010000:00010000:1.0:1788572540.344812:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572540.344821:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.344830:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572540.344835:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.344856:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.344870:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x77:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572540.344873:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572540.344877:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 1/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.344884:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161a5b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x77:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.344903:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572540.344905:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572540.344907:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572540.348132:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348149:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348185:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348197:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348205:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348211:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.348221:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348229:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348248:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:1.0:1788572540.348251:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572540.348256:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.348263:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161a38 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.348395:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35892:x1875453360285312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 33424us (33591us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572540.348405:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3876 00000100:00100000:2.0:1788572540.350059:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f90a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382339584:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.350087:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.356418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.356585:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360286848 00000100:00100000:0.0:1788572540.356596:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360286848 00010000:00010000:0.0:1788572540.356638:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.356685:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 923 00000100:00100000:0.0:1788572540.356696:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360286848:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572540.356711:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.356718:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.356731:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.356755:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16192e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x76:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef466 expref: 70 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.356771:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:0.0:1788572540.356776:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.356786:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.356793:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.356828:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.356892:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2349:x1875453360286848:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 197us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.356899:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 923 00010000:00010000:2.0:1788572540.357442:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357452:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357493:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357501:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357507:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357514:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.357518:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357527:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357541:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:2.0:1788572540.357544:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572540.357547:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.357553:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161a62 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.357613:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35893:x1875453360286464:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 14051us (16373us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572540.357623:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3879 00000100:00100000:2.0:1788572540.359951:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.360040:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360286976 00000100:00100000:1.0:1788572540.360050:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360286976 00000100:00100000:1.0:1788572540.360105:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 993 00000100:00100000:1.0:1788572540.360116:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360286976:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.360160:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.360183:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.360192:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.360197:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.360203:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.360249:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.360265:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 12 pid: 20911 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.360280:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382339712:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.360541:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec040780 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.364142:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382339712:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.364174:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.366107:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.366212:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360287104 00000100:00100000:3.0:1788572540.366233:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360287104 00000100:00100000:3.0:1788572540.366287:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 994 00000100:00100000:3.0:1788572540.366297:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360287104:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.366353:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.366375:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.366385:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.366389:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572540.366395:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.366433:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.366447:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 11 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.366474:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382339840:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572540.366563:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.370820:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382339840:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.370850:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.372251:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.372342:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.372476:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360287232 00000100:00100000:1.0:1788572540.372484:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360287232 00010000:00010000:1.0:1788572540.372525:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 13 pid: 20911 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.372570:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360287360 00000100:00100000:1.0:1788572540.372574:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360287360 00010000:00010000:1.0:1788572540.372589:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 12 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.372624:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 924 00000100:00100000:1.0:1788572540.372634:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360287232:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.372649:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.372673:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 13 pid: 20911 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372686:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 13 pid: 20911 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372709:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb161a15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4d6 expref: 13 pid: 20911 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372727:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.372731:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.372737:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372748:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372781:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.372830:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360287232:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 197us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.372838:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 924 00000100:00100000:1.0:1788572540.372868:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 925 00000100:00100000:1.0:1788572540.372875:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360287360:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.372884:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.372898:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 12 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372909:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 12 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372923:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b1d26361/0x85ace67feb1619dd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x42:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4ac expref: 12 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372939:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.372942:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.372946:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372953:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.372969:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.372990:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360287360:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 115us (648us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.372999:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 925 00010000:00010000:1.0:1788572540.373043:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373053:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373077:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373085:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373092:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373098:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e76ea200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572540.373102:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373112:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373123:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.373135:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059e23ee6/0x85ace67feb161a69 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.373178:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360286976:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 13065us (13247us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.373185:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 993 00010000:00010000:3.0:1788572540.373330:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373339:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373357:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373365:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373372:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373378:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572540.373380:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373387:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373398:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.373405:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb161a70 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.373451:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360287104:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 7156us (7355us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.373458:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 994 00000100:00100000:0.0:1788572540.379564:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.379761:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360287616 00000100:00100000:2.0:1788572540.379772:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360287616 00000100:00100000:2.0:1788572540.379833:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3880 00000100:00100000:2.0:1788572540.379843:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35892:x1875453360287616:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.379953:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.379971:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.379983:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.380267:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.380280:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.380288:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 4/0,1 mode: --/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.380841:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.380908:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 69 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.380925:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382339968:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.381170:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.385720:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.388641:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360287872 00000100:00100000:1.0:1788572540.388656:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360287872 00000100:00100000:1.0:1788572540.388727:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3881 00000100:00100000:1.0:1788572540.388738:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35893:x1875453360287872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.388825:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.388841:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.388853:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.389138:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.389150:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.389159:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 4/0,1 mode: --/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.389200:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.389213:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 71 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.389228:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382340096:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.389381:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec040780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.390747:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382339968:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.390782:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.391171:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.391252:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360288000 00000100:00100000:3.0:1788572540.391260:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360288000 00010000:00010000:3.0:1788572540.391295:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 72 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.391342:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 926 00000100:00100000:3.0:1788572540.391352:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e53050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2350:x1875453360288000:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.391367:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.391373:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 72 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.391385:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 72 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.391406:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1619c8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef49e expref: 72 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.391423:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:3.0:1788572540.391429:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.391439:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.391447:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.391486:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572540.391541:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e53050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2350:x1875453360288000:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 190us (371us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.391550:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 926 00010000:00010000:2.0:1788572540.392122:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.392131:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572540.392345:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970216 00000001:00080000:2.0:1788572540.392533:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970218 00000001:00080000:2.0:1788572540.392581:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970219 00000100:00100000:2.0:1788572540.392670:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382340224:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572540.392729:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572540.392775:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.392923:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382340224 00000100:00100000:0.0:1788572540.392931:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382340224 00000100:00100000:0.0:1788572540.392984:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1788572540.392993:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382340224:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572540.393104:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382340224:12345-0@lo:1000:mdt00_003.0 Request processed in 112us (374us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.393112:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:2.0:1788572540.393481:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382340224:0@lo:1000:mdt00_003.0 00000001:00080000:2.0:1788572540.393842:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970220 00000100:00100000:2.0:1788572540.393905:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382340352:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572540.393942:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572540.393978:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb56c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.394115:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382340352 00000100:00100000:1.0:1788572540.394122:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382340352 00000100:00100000:1.0:1788572540.394162:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1788572540.394170:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382340352:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572540.394372:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:9040:x1875453382340352:12345-0@lo:1000:mdt00_003.0 Request processed in 202us (428us total) trans 4294967553 rc 0/0 00000100:00100000:1.0:1788572540.394380:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:2.0:1788572540.395769:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382340352:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1788572540.395816:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395828:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395836:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395842:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.395847:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395855:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395870:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627e7451800) 00010000:00010000:2.0:1788572540.395874:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.395879:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395886:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161a7e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x75:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395921:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395928:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395946:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395952:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.395956:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395962:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395970:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572540.395973:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.395975:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.395981:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb15ed52 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396034:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396049:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396058:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396066:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396072:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396078:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396082:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.396085:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396091:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396098:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572540.396101:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.396104:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396110:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161a93 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396118:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396124:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396130:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396135:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.396138:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396147:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396155:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572540.396158:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572540.396161:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.396167:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161a77 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.396694:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35892:x1875453360287616:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 16848us (17137us total) trans 4294970221 rc 0/0 00000100:00100000:2.0:1788572540.396705:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3880 00000020:00080000:2.0:1788572540.396711:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572540.400899:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.401035:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360288256 00000100:00100000:3.0:1788572540.401046:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360288256 00000100:00100000:3.0:1788572540.401105:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 858 00000100:00100000:3.0:1788572540.401115:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e67450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35892:x1875453360288256:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572540.401639:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x75:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572540.401683:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970222 00010000:00010000:3.0:1788572540.401738:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.401756:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.401769:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.401778:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.401789:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.401795:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.401825:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572540.401925:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e67450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35892:x1875453360288256:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 810us (1039us total) trans 4294970223 rc 0/0 00000100:00100000:3.0:1788572540.401934:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 858 00000020:00080000:3.0:1788572540.401939:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572540.402184:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572540.402192:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.402199:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572540.402204:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.402225:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.402240:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x75:0x0].0x0 (ffff9627d599a900) 00010000:00010000:2.0:1788572540.402243:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572540.402249:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 1/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.402259:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161a9a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x75:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.402278:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572540.402280:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572540.402282:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572540.409133:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.409399:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360288384 00000100:00100000:3.0:1788572540.409410:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360288384 00010000:00010000:3.0:1788572540.409455:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 76 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.409558:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382340096:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.409585:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.409643:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 927 00000100:00100000:3.0:1788572540.409654:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2349:x1875453360288384:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.409670:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.409676:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 76 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.409688:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 76 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.409708:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161951 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef474 expref: 76 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.409730:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:3.0:1788572540.409736:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.409745:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.409753:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.409785:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572540.409841:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360288384:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 189us (719us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.409862:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 927 00010000:00010000:1.0:1788572540.409912:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.409920:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572540.410154:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970217 00000001:00080000:1.0:1788572540.411259:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970224 00000001:00080000:1.0:1788572540.411437:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970225 00000100:00100000:1.0:1788572540.411547:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382340480:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572540.411607:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572540.411692:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f376c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.413413:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382340480 00000100:00100000:2.0:1788572540.413429:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382340480 00000100:00100000:2.0:1788572540.413566:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1788572540.413578:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382340480:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572540.413732:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382340480:12345-0@lo:1000:mdt00_004.0 Request processed in 156us (2123us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.413739:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1788572540.413787:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967546 gen 2 00000004:00080000:2.0:1788572540.413793:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967545 committed transno 4294967546 00000004:00080000:2.0:1788572540.413814:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967546 committed transno 4294967546 00000100:00100000:2.0:1788572540.413830:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382340480:0@lo:1000:mdt00_004.0 00000001:00080000:2.0:1788572540.414613:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970226 00000100:00100000:2.0:1788572540.414704:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786df80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382340608:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572540.414760:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572540.414833:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3251480 going to sleep for 11 seconds 00000100:00080000:1.0:1788572540.415716:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788572540.415806:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382340608 00000100:00100000:3.0:1788572540.415813:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382340608 00000100:00100000:3.0:1788572540.415894:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:3.0:1788572540.415903:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2f50450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382340608:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572540.416424:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2f50450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382340608:12345-0@lo:1000:mdt00_004.0 Request processed in 522us (1663us total) trans 4294967550 rc 0/0 00000100:00100000:3.0:1788572540.416434:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:1.0:1788572540.416904:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786df80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382340608:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572540.417140:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417151:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417158:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417164:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e76eba00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.417168:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417176:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417192:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d599a800) 00010000:00010000:1.0:1788572540.417195:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572540.417200:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417207:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161a8c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x76:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417369:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417378:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417384:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417389:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.417391:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417398:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417407:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:1.0:1788572540.417409:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572540.417412:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417417:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb15ed98 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417440:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417453:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417473:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417481:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417489:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417494:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417500:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.417502:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417508:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417516:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:1.0:1788572540.417518:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.417521:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417526:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161aa1 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417536:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417542:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417548:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417553:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e76ea200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.417557:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417562:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417569:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:1.0:1788572540.417572:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572540.417575:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.417581:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161a85 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.417813:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35893:x1875453360287872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 29076us (32102us total) trans 4294970227 rc 0/0 00000100:00100000:1.0:1788572540.417824:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3881 00000020:00080000:1.0:1788572540.417828:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:0.0:1788572540.420870:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572540.420880:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572540.420950:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 124 count 101 00000040:00080000:0.0:1788572540.420959:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:0.0:1788572540.421129:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ca8caa00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382340736:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572540.421225:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628021586c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.426176:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.426512:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360288640 00000100:00100000:1.0:1788572540.426521:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360288640 00000100:00100000:1.0:1788572540.426713:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 859 00000100:00100000:1.0:1788572540.426724:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35893:x1875453360288640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572540.427252:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x76:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572540.427294:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970228 00010000:00010000:1.0:1788572540.427353:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.427369:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.427381:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.427391:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.427398:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.427405:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.427430:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572540.427910:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ca8caa00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382340736:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572540.428129:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.428153:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360288768 00000100:00100000:0.0:1788572540.428159:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360288768 00000100:00100000:0.0:1788572540.428198:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3882 00000100:00100000:0.0:1788572540.428205:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35896:x1875453360288768:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572540.428237:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbe:0x0] from 18446744073709551615 #0 00010000:00010000:2.0:1788572540.429194:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572540.429205:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.429215:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572540.429231:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.429240:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.429255:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x76:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:2.0:1788572540.429258:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572540.429263:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.429269:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161aa8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x76:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.429287:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572540.429289:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572540.429291:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572540.429439:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35896:x1875453360288768:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 1231us (1307us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.429449:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3882 00000100:00100000:2.0:1788572540.430202:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.430291:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.431400:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360288896 00000100:00100000:0.0:1788572540.431411:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360288896 00000100:00100000:0.0:1788572540.431461:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360289024 00000100:00100000:0.0:1788572540.431475:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360289024 00000100:00100000:0.0:1788572540.431512:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3883 00000100:00100000:0.0:1788572540.431522:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35894:x1875453360288896:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572540.431555:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbb:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572540.431655:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35894:x1875453360288896:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 135us (1455us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.431663:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3883 00000100:00100000:0.0:1788572540.431679:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3884 00000100:00100000:0.0:1788572540.431685:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35895:x1875453360289024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572540.431700:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xc1:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572540.431741:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35895:x1875453360289024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 57us (1450us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.431748:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3884 00000100:00100000:1.0:1788572540.436985:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35893:x1875453360288640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 10250us (10806us total) trans 4294970229 rc 0/0 00000100:00100000:1.0:1788572540.437019:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 859 00000020:00080000:1.0:1788572540.437027:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788572540.440151:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.440186:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.440411:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360289280 00000100:00100000:0.0:1788572540.440421:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360289280 00000100:00100000:0.0:1788572540.440484:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360289152 00000100:00100000:0.0:1788572540.440488:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360289152 00000100:00100000:0.0:1788572540.440510:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3885 00000100:00100000:0.0:1788572540.440519:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35895:x1875453360289280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.440598:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440616:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440628:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440644:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440652:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440658:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440684:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440692:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440699:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.440736:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc1:0x0]: returning size 3 00000004:00200000:0.0:1788572540.440740:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.440753:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440765:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440771:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440780:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440786:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.440791:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440799:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440811:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.440814:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.440818:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440824:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161aaf lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440836:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440842:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440864:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440870:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.440873:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440879:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440886:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x31536349 (ffff9627e7451200) 00010000:00010000:0.0:1788572540.440889:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.440892:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440898:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ab6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x31536349 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440915:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440921:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440927:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.440933:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.441150:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441158:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441169:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:0.0:1788572540.441171:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.441175:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441180:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161abd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.441256:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35895:x1875453360289280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 737us (1115us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.441263:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3885 00000100:00100000:0.0:1788572540.441283:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3886 00000100:00100000:0.0:1788572540.441291:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35896:x1875453360289152:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.441392:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441403:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441410:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441423:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441430:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441436:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441453:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441473:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441480:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.441503:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbe:0x0]: returning size 3 00000004:00200000:0.0:1788572540.441506:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.441512:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441520:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441526:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441532:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441536:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.441539:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441545:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441554:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.441556:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.441559:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441564:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161ac4 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441574:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441580:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441585:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441591:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.441593:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441599:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441607:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xa289902d (ffff9627e7451400) 00010000:00010000:0.0:1788572540.441609:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.441612:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441618:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161acb lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xa289902d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441629:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441636:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441641:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441645:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.441648:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441654:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441660:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627d599af00) 00010000:00010000:0.0:1788572540.441662:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.441665:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.441670:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161ad2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.441704:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35896:x1875453360289152:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 413us (1516us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.441711:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3886 00000100:00100000:0.0:1788572540.444647:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.444709:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360289408 00000100:00100000:2.0:1788572540.444718:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360289408 00000100:00100000:2.0:1788572540.444805:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3887 00000100:00100000:2.0:1788572540.444815:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35894:x1875453360289408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.444902:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.444921:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.444933:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.444964:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.444972:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.444978:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445028:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445037:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445043:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.445080:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbb:0x0]: returning size 3 00000004:00200000:2.0:1788572540.445084:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.445096:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445107:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445113:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445120:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445129:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.445133:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445140:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445151:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572540.445154:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.445159:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445166:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb161ad9 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445178:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445184:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445190:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445195:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.445198:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445204:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445225:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x99c4a514 (ffff9627e7a95900) 00010000:00010000:2.0:1788572540.445228:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.445231:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445237:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb161ae0 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x99c4a514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445251:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445256:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445261:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445266:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.445269:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445275:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445283:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572540.445285:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.445288:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.445293:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb161ae7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.445630:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35894:x1875453360289408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 815us (991us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.445638:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3887 00000100:00100000:1.0:1788572540.451149:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.453128:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.453161:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360289664 00000100:00100000:0.0:1788572540.453173:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360289664 00000100:00100000:3.0:1788572540.453549:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3888 00000100:00100000:3.0:1788572540.453562:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35895:x1875453360289664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.453585:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.453607:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161aee lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4dd expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453715:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453731:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453743:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572540.453754:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc1:0x0]: returning size 3 00010000:00010000:3.0:1788572540.453857:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453865:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453872:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453878:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572540.453883:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453910:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453923:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:3.0:1788572540.453926:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572540.453930:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453948:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161af5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.453994:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161aee lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4dd expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.454019:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572540.454022:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572540.454026:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161aee lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4dd expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.454035:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679800, rc 0) 00000100:00100000:3.0:1788572540.454096:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35895:x1875453360289664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 537us (2957us total) trans 4294970230 rc 301/301 00000100:00100000:3.0:1788572540.454104:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3888 00000100:00100000:1.0:1788572540.457842:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.457893:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.457947:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360289920 00000100:00100000:0.0:1788572540.457957:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360289920 00000100:00100000:0.0:1788572540.458000:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360290304 00000100:00100000:0.0:1788572540.458031:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360290304 00000100:00100000:0.0:1788572540.458049:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360290176 00000100:00100000:0.0:1788572540.458052:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360290176 00000100:00100000:0.0:1788572540.458070:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3889 00000100:00100000:0.0:1788572540.458079:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35896:x1875453360289920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.458101:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.458122:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161b03 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4e4 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.458238:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.458254:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.458266:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.458278:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbe:0x0]: returning size 3 00000100:00100000:2.0:1788572540.458882:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3890 00000100:00100000:2.0:1788572540.458896:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35895:x1875453360290304:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572540.458921:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xc1:0x0] from 18446744073709551615 #0 00010000:00010000:0.0:1788572540.459150:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459160:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459167:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459173:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.459177:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459189:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459203:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627d599af00) 00010000:00010000:0.0:1788572540.459206:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572540.459221:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459229:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459275:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161b03 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4e4 expref: 85 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459286:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.459288:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572540.459292:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161b03 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4e4 expref: 85 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.459312:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bace00, rc 0) 00000100:00100000:2.0:1788572540.459890:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35895:x1875453360290304:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 994us (2052us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.459900:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3890 00000100:00100000:2.0:1788572540.459930:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3891 00000100:00100000:2.0:1788572540.459954:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35894:x1875453360290176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.459966:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.459981:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161b18 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4eb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460075:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460088:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460098:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.460108:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbb:0x0]: returning size 3 00010000:00010000:2.0:1788572540.460174:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460183:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460191:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460196:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.460200:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460206:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460229:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572540.460232:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572540.460236:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460242:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb161b1f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460273:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161b18 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4eb expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460282:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572540.460284:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572540.460288:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb161b18 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4eb expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.460296:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05a00, rc 0) 00000100:00100000:0.0:1788572540.460641:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35896:x1875453360289920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2561us (7519us total) trans 4294970231 rc 301/301 00000100:00100000:0.0:1788572540.460650:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3889 00000100:00100000:2.0:1788572540.467664:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35894:x1875453360290176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 7702us (9756us total) trans 4294970232 rc 301/301 00000100:00100000:2.0:1788572540.467677:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3891 00000100:00100000:3.0:1788572540.467798:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.467987:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.470623:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360290432 00000100:00100000:2.0:1788572540.470637:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360290432 00000100:00100000:2.0:1788572540.470701:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360290560 00000100:00100000:2.0:1788572540.470705:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360290560 00000100:00100000:2.0:1788572540.470732:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3892 00000100:00100000:2.0:1788572540.470740:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35895:x1875453360290432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.470761:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.470783:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161b2d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4f2 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.470865:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.470881:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.470893:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.470908:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.470914:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.470920:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471551:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161b42 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471562:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161b42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471570:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161b42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.471600:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:2.0:1788572540.471604:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.471623:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161b42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471635:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471642:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471649:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471655:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.471660:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471667:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471681:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572540.471684:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.471689:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471695:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb161b34 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471709:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471715:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471720:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471725:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.471727:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471732:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471739:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627e7a95b00) 00010000:00010000:2.0:1788572540.471742:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.471744:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471750:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb161b3b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471783:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb161b2d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4f2 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471796:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161b42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4f2 expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.471803:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572540.471805:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572540.471808:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:3.0:1788572540.471995:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3893 00000100:00100000:3.0:1788572540.472022:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35896:x1875453360290560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572540.472048:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbe:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572540.472677:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35895:x1875453360290432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1934us (4877us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.472686:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3892 00000100:00100000:1.0:1788572540.472788:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.473247:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360290688 00000100:00100000:1.0:1788572540.473255:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360290688 00000100:00100000:1.0:1788572540.473291:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3894 00000100:00100000:1.0:1788572540.473310:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35894:x1875453360290688:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572540.473334:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbb:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572540.473400:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35894:x1875453360290688:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 101us (612us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.473408:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3894 00000100:00100000:1.0:1788572540.476704:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.476839:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360290944 00000100:00100000:2.0:1788572540.476849:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360290944 00000100:00100000:2.0:1788572540.476914:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3895 00000100:00100000:2.0:1788572540.476924:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35895:x1875453360290944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.476953:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.476957:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572540.476985:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb161b42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4f2 expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.477035:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572540.477042:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572540.477058:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161b49 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef4f9 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572540.477779:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x43:0x0], index 1: 0 00010000:00010000:2.0:1788572540.477964:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161b50 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.477981:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161b50 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.477991:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161b50 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.478090:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161b49 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4f9 expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.478105:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161b50 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4f9 expref: 86 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.478113:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7a95d00) 00010000:00010000:2.0:1788572540.478116:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572540.478119:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:2.0:1788572540.478187:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35895:x1875453360290944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1263us (1492us total) trans 4294970233 rc 0/0 00000100:00100000:2.0:1788572540.478196:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3895 00000100:00100000:3.0:1788572540.479857:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35896:x1875453360290560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 7835us (11852us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.479871:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3893 00000100:00100000:3.0:1788572540.484190:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.484439:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360291072 00000100:00100000:1.0:1788572540.484450:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360291072 00000100:00100000:1.0:1788572540.484525:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3896 00000100:00100000:1.0:1788572540.484537:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35894:x1875453360291072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.484560:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572540.484583:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161b5e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef500 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.484657:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.484671:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.484682:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.484700:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.484706:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.484711:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485048:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161b73 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485058:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161b73 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485066:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161b73 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572540.485094:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:1.0:1788572540.485098:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572540.485117:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161b73 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485129:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485136:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485143:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485150:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e76eba00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.485154:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485163:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485176:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572540.485179:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.485184:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485190:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb161b65 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485206:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485212:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485510:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485516:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.485519:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485525:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485534:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627d0ee2000) 00010000:00010000:1.0:1788572540.485536:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.485540:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485546:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161b6c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485580:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161b5e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef500 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485595:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161b73 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef500 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.485608:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572540.485610:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572540.485613:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715800, rc 0) 00000100:00100000:3.0:1788572540.485927:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35894:x1875453360291072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1390us (1745us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.485946:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3896 00000100:00100000:3.0:1788572540.491291:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.491897:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360291328 00000100:00100000:0.0:1788572540.491911:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360291328 00000100:00100000:0.0:1788572540.491986:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3897 00000100:00100000:0.0:1788572540.491996:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35896:x1875453360291328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.492048:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.492052:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572540.492078:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161b73 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef500 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.492097:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572540.492102:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.492116:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b7a lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef507 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572540.492754:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x44:0x0], index 0: 0 00010000:00010000:0.0:1788572540.492954:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161b81 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.492969:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161b81 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.492980:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161b81 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.493557:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161b7a lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef507 expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.493575:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161b81 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef507 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.493582:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451400) 00010000:00010000:0.0:1788572540.493585:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572540.493588:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246d800, rc 0) 00000100:00100000:0.0:1788572540.493660:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35896:x1875453360291328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1664us (2378us total) trans 4294970234 rc 0/0 00000100:00100000:0.0:1788572540.493668:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3897 00000100:00100000:1.0:1788572540.497631:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.497757:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360291456 00000100:00100000:2.0:1788572540.497768:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360291456 00000100:00100000:2.0:1788572540.497832:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3898 00000100:00100000:2.0:1788572540.497843:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35895:x1875453360291456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.497873:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.497895:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161b8f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef50e expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.497975:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.497991:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498173:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498194:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498202:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498210:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498643:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ba4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498653:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ba4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498660:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ba4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.498688:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:2.0:1788572540.498691:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.498709:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ba4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498720:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498732:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498738:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498744:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.498748:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498755:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498768:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572540.498771:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.498776:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498782:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161b96 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498794:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498800:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498806:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498811:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.498814:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498820:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498829:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627e7a95b00) 00010000:00010000:2.0:1788572540.498831:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.498833:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498838:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161b9d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498872:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161b8f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef50e expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498885:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ba4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef50e expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.498892:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572540.498894:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572540.498896:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53000, rc 0) 00000100:00100000:2.0:1788572540.501188:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35895:x1875453360291456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3333us (3552us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.501201:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3898 00000100:00100000:1.0:1788572540.502311:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.502706:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360291712 00000100:00100000:2.0:1788572540.502716:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360291712 00000100:00100000:2.0:1788572540.502783:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3899 00000100:00100000:2.0:1788572540.502792:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35894:x1875453360291712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.502820:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.502841:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161bab lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef515 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572540.503534:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x44:0x0], index 1: 0 00010000:00010000:2.0:1788572540.503730:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161bb2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.503745:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161bb2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.503757:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161bb2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.503819:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161bab lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef515 expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.503839:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161bb2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef515 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.504130:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572540.504134:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572540.504138:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:2.0:1788572540.504210:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35894:x1875453360291712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1417us (1914us total) trans 4294970235 rc 0/0 00000100:00100000:2.0:1788572540.504218:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3899 00000100:00100000:0.0:1788572540.509775:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.509980:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360291968 00000100:00100000:1.0:1788572540.509991:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360291968 00000100:00100000:1.0:1788572540.510140:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3900 00000100:00100000:1.0:1788572540.510151:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35894:x1875453360291968:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.510211:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572540.510402:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161bb2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef515 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510422:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:1.0:1788572540.510439:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ba4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef50e expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510447:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572540.510450:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572540.510483:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510497:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510507:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510521:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510529:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510536:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510732:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510741:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510748:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510754:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.510758:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510766:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510775:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572540.510778:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.510781:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510786:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bc0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510795:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510801:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510806:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510811:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.510814:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510820:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510827:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xa1238770 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572540.510829:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.510832:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.510838:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bc7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0xa1238770 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.511073:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35894:x1875453360291968:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 923us (1307us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572540.511082:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3900 00000100:00100000:3.0:1788572540.518661:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.518709:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.518830:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292096 00000100:00100000:1.0:1788572540.518841:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292096 00000100:00100000:1.0:1788572540.518896:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292352 00000100:00100000:1.0:1788572540.518900:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292352 00000100:00100000:1.0:1788572540.518928:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3901 00000100:00100000:1.0:1788572540.518938:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35894:x1875453360292096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.518960:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572540.518984:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bce lrc: 2/0,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef51c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.519044:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.519060:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.519073:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.519179:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161bce lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.519198:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 90 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.519207:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572540.519209:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572540.519224:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:1.0:1788572540.519276:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35894:x1875453360292096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 339us (625us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.519283:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3901 00000100:00100000:3.0:1788572540.519395:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3902 00000100:00100000:3.0:1788572540.519405:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35895:x1875453360292352:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.519437:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.519453:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161b50 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef4f9 expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519476:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7a95d00) 00010000:00010000:3.0:1788572540.519485:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572540.519501:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519511:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519519:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519532:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519540:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519547:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519736:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519745:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519751:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519757:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572540.519762:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519770:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519780:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572540.519782:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572540.519787:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519794:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161bdc lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519803:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519809:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519815:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519821:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572540.519824:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519830:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519837:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x1873d186 (ffff9627d599a400) 00010000:00010000:3.0:1788572540.519840:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572540.519842:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.519848:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161be3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x1873d186 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.519908:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35895:x1875453360292352:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 505us (1198us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572540.519914:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3902 00000100:00100000:3.0:1788572540.526094:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.526529:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292480 00000100:00100000:0.0:1788572540.526544:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292480 00000100:00100000:0.0:1788572540.526612:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3903 00000100:00100000:0.0:1788572540.526623:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35895:x1875453360292480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.526648:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.526675:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161bea lrc: 2/0,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef523 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.526715:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.526729:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.526741:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.526855:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161bea lrc: 0/0,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.526875:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.526884:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:0.0:1788572540.526888:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572540.526891:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bace00, rc 0) 00000100:00100000:0.0:1788572540.526964:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35895:x1875453360292480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 343us (880us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.526971:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3903 00000100:00100000:0.0:1788572540.530235:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.530269:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.530295:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.530336:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.530355:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.530378:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.530403:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292736 00000100:00100000:0.0:1788572540.530413:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292736 00000100:00100000:0.0:1788572540.530460:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360293248 00000100:00100000:0.0:1788572540.530474:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360293248 00000100:00100000:0.0:1788572540.530493:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360293504 00000100:00100000:0.0:1788572540.530496:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360293504 00000100:00100000:0.0:1788572540.530516:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 628 00000100:00100000:0.0:1788572540.530526:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:2347:x1875453360292736:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572540.530607:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:2347:x1875453360292736:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 82us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.530615:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 628 00000100:00100000:0.0:1788572540.530633:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3904 00000100:00100000:0.0:1788572540.530639:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35896:x1875453360293248:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.530690:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.530718:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161b81 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef507 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.530744:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451400) 00010000:00010000:0.0:1788572540.530755:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.530778:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.530792:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.530802:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.530817:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.530825:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.530832:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531077:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531086:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531093:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531098:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.531104:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531112:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531122:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572540.531125:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.531128:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531134:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161bf8 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531143:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531149:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531155:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531160:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.531163:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531168:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531174:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x5a49a372 (ffff9627e7451600) 00010000:00010000:0.0:1788572540.531176:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.531179:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531190:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161bff lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x5a49a372 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.531231:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35896:x1875453360293248:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 591us (874us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572540.531237:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3904 00000100:00100000:0.0:1788572540.531250:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3905 00000100:00100000:0.0:1788572540.531257:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35894:x1875453360293504:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.531299:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531309:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.531316:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572540.531634:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].53, 88 off12448 00000004:00080000:0.0:1788572540.531643:0:5837:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 888600638007ns, next at 889600635412ns, rc = 0 00000004:00010000:0.0:1788572540.531727:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d90952aa reply state 000000002419e34a transno 4294970236 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.531737:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.531792:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35894:x1875453360293504:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 533us (1410us total) trans 4294970236 rc 0/0 00000100:00100000:0.0:1788572540.531801:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3905 00000100:00100000:1.0:1788572540.532288:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292608 00000100:00100000:1.0:1788572540.532296:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292608 00000100:00100000:1.0:1788572540.532342:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 677 00000100:00100000:1.0:1788572540.532351:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2347:x1875453360292608:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572540.532355:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292864 00000100:00100000:2.0:1788572540.532359:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292864 00000100:00100000:1.0:1788572540.532386:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2347:x1875453360292608:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 37us (2162us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.532392:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 677 00000100:00100000:2.0:1788572540.532395:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360292992 00000100:00100000:2.0:1788572540.532398:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360292992 00000100:00100000:2.0:1788572540.532419:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 598 00000100:00100000:2.0:1788572540.532427:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+31:2347:x1875453360292864:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572540.532456:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+31:2347:x1875453360292864:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 30us (2186us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.532493:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 598 00000100:00100000:2.0:1788572540.532510:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 599 00000100:00100000:2.0:1788572540.532517:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2347:x1875453360292992:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572540.532541:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2347:x1875453360292992:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 25us (2205us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.532547:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 599 00000100:00080000:3.0:1788572540.533372:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788572540.533382:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572540.538670:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.539490:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360293760 00000100:00100000:2.0:1788572540.539503:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360293760 00000100:00100000:2.0:1788572540.539570:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3906 00000100:00100000:2.0:1788572540.539581:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35895:x1875453360293760:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.539679:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.539699:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.539711:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572540.540125:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].54, 88 off12536 00000004:00010000:2.0:1788572540.540223:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006bc319a3 reply state 000000006cd30376 transno 4294970237 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.540237:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572540.540340:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35895:x1875453360293760:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 760us (1678us total) trans 4294970237 rc 0/0 00000100:00100000:2.0:1788572540.540347:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3906 00000100:00100000:3.0:1788572540.541132:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.541186:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360293888 00000100:00100000:0.0:1788572540.541194:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360293888 00000100:00100000:3.0:1788572540.541206:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.541239:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360294016 00000100:00100000:0.0:1788572540.541243:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360294016 00000100:00100000:3.0:1788572540.541260:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3907 00000100:00100000:3.0:1788572540.541282:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35894:x1875453360293888:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.541301:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.541318:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef52a expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.541372:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.541388:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.541407:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:3.0:1788572540.541410:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572540.541415:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678400, rc 0) 00000100:00100000:3.0:1788572540.541453:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35894:x1875453360293888:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 184us (325us total) trans 0 rc 0/0 00010000:00010000:1.0:1788572540.541549:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.541558:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.541565:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.541571:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.541576:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.541584:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.541596:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572540.541599:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572540.541602:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.541609:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161c06 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.541690:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3908 00000100:00100000:0.0:1788572540.541701:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35896:x1875453360294016:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.541713:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.541726:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161c1b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef531 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.541749:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.541758:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.541767:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.541862:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161c1b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.541877:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 94 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.541885:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:0.0:1788572540.541888:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572540.541891:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bac400, rc 0) 00000100:00100000:0.0:1788572540.542090:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35896:x1875453360294016:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 391us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.542101:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3908 00000100:00100000:1.0:1788572540.556783:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.556818:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.556891:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.556923:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360294528 00000100:00100000:1.0:1788572540.556940:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360294528 00000100:00100000:1.0:1788572540.556997:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 995 00000100:00100000:1.0:1788572540.557023:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360294528:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572540.558571:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360294528:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 1543us (1673us total) trans 4294967551 rc 0/0 00000100:00100000:1.0:1788572540.558584:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 995 00000100:00100000:3.0:1788572540.559282:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3907 00000100:00100000:3.0:1788572540.559332:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360294144 00000100:00100000:3.0:1788572540.559339:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360294144 00000100:00100000:3.0:1788572540.559376:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360294400 00000100:00100000:3.0:1788572540.559380:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360294400 00000100:00100000:3.0:1788572540.559405:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3909 00000100:00100000:3.0:1788572540.559413:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35895:x1875453360294144:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.559434:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.559452:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef538 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.559522:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.559535:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.559553:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:3.0:1788572540.559556:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572540.559559:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679400, rc 0) 00000100:00100000:3.0:1788572540.559588:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35895:x1875453360294144:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 176us (2817us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.559593:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3909 00000100:00100000:3.0:1788572540.559606:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3910 00000100:00100000:3.0:1788572540.559612:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35896:x1875453360294400:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.559671:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.559680:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.559688:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572540.560061:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].56, 88 off12640 00000004:00080000:3.0:1788572540.560070:0:5839:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 888629065184ns, next at 889629063206ns, rc = 0 00000004:00010000:3.0:1788572540.560149:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b134f949 reply state 00000000c12902a1 transno 4294970238 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.560157:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572540.560210:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35896:x1875453360294400:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 598us (3391us total) trans 4294970238 rc 0/0 00000100:00100000:3.0:1788572540.560216:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3910 00010000:00010000:2.0:1788572540.561418:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.561430:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.561438:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.561444:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.561448:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.561456:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.561481:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:2.0:1788572540.561484:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572540.561489:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.561495:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c0d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.569484:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572540.569596:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.569600:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360294656 00010000:00010000:1.0:1788572540.569608:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.569609:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360294656 00010000:00010000:1.0:1788572540.569615:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.569621:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.569626:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.569634:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.569651:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:1.0:1788572540.569654:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572540.569659:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.569668:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 996 00010000:00010000:1.0:1788572540.569670:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c30 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.569677:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360294656:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.569990:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360294656:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 313us (696us total) trans 4294967552 rc 0/0 00000100:00100000:2.0:1788572540.569999:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 996 00000100:00100000:0.0:1788572540.573780:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.574081:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360294912 00000100:00100000:1.0:1788572540.574092:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360294912 00000100:00100000:1.0:1788572540.574144:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 860 00000100:00100000:1.0:1788572540.574154:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35894:x1875453360294912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.574442:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.574510:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360295040 00000100:00100000:2.0:1788572540.574515:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360295040 00000100:00100000:2.0:1788572540.574560:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3911 00000100:00100000:2.0:1788572540.574568:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35896:x1875453360295040:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.574583:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.574603:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef53f expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.574650:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 96 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.574666:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 96 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.574686:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:2.0:1788572540.574690:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.574694:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:2.0:1788572540.574761:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35896:x1875453360295040:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 193us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.574768:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3911 00000100:00100000:1.0:1788572540.575625:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35894:x1875453360294912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1468us (1847us total) trans 4294970239 rc 0/0 00000100:00100000:1.0:1788572540.575635:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 860 00000100:00100000:1.0:1788572540.576931:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.577048:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360295296 00000100:00100000:2.0:1788572540.577056:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360295296 00000100:00100000:2.0:1788572540.577108:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 861 00000100:00100000:2.0:1788572540.577118:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35895:x1875453360295296:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.577626:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35895:x1875453360295296:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 507us (698us total) trans 4294970240 rc 0/0 00000100:00100000:2.0:1788572540.577634:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 861 00000100:00100000:1.0:1788572540.583449:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.585334:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.585593:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.585662:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360295424 00000100:00100000:2.0:1788572540.585673:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360295424 00000100:00100000:2.0:1788572540.585732:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 997 00000100:00100000:2.0:1788572540.585747:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7661450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360295424:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.585942:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360295680 00000100:00100000:3.0:1788572540.585947:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360295680 00000100:00100000:3.0:1788572540.586028:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360295936 00000100:00100000:3.0:1788572540.586031:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360295936 00000100:00100000:0.0:1788572540.586079:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3912 00000100:00100000:2.0:1788572540.586080:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7661450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360295424:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 339us (756us total) trans 4294967554 rc 0/0 00000100:00100000:0.0:1788572540.586088:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35895:x1875453360295680:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.586088:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 997 00000100:00100000:0.0:1788572540.586151:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35895:x1875453360295680:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 65us (2713us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.586156:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3912 00000100:00100000:0.0:1788572540.586168:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3913 00000100:00100000:0.0:1788572540.586174:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35894:x1875453360295936:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572540.586200:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35894:x1875453360295936:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 27us (609us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.586205:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3913 00000100:00100000:1.0:1788572540.591205:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.591247:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.591282:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.591402:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360296064 00000100:00100000:2.0:1788572540.591413:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360296064 00000100:00100000:3.0:1788572540.591475:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360296192 00000100:00100000:3.0:1788572540.591480:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360296192 00000100:00100000:2.0:1788572540.591483:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3914 00000100:00100000:2.0:1788572540.591492:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35895:x1875453360296064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.591508:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3915 00000100:00100000:3.0:1788572540.591514:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35894:x1875453360296192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.591515:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572540.591521:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.591535:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161c3e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef546 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.591535:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161c45 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef54d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.591602:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161c4c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.591602:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c53 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.591618:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161c4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.591618:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c53 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.591632:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161c4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.591632:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c53 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.591668:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00200000:3.0:1788572540.591668:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.591684:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161c4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572540.591684:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c53 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.591711:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161c3e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef546 expref: 102 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.591711:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb161c45 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef54d expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.591730:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161c4c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef546 expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.591730:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c53 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef54d expref: 100 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.591739:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:3.0:1788572540.591739:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:2.0:1788572540.591743:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572540.591743:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572540.591746:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678200, rc 0) 00010000:00010000:2.0:1788572540.591747:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52c00, rc 0) 00000100:00100000:2.0:1788572540.591800:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35895:x1875453360296064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 309us (605us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.591808:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3914 00000100:00100000:0.0:1788572540.591870:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360296448 00000100:00100000:0.0:1788572540.591875:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360296448 00000100:00100000:0.0:1788572540.591910:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 862 00000100:00100000:0.0:1788572540.591917:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35896:x1875453360296448:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.592786:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35894:x1875453360296192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1267us (1533us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.592795:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3915 00000100:00100000:0.0:1788572540.592869:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35896:x1875453360296448:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 951us (1586us total) trans 4294970241 rc 0/0 00000100:00100000:0.0:1788572540.592877:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 862 00000100:00100000:0.0:1788572540.599688:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.599735:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.599811:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.599877:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360296960 00000100:00100000:2.0:1788572540.599888:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360296960 00000100:00100000:2.0:1788572540.599944:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360296832 00000100:00100000:2.0:1788572540.599948:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360296832 00000100:00100000:3.0:1788572540.599979:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360296704 00000100:00100000:3.0:1788572540.599983:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360296704 00000100:00100000:3.0:1788572540.600037:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3916 00000100:00100000:3.0:1788572540.600047:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35896:x1875453360296704:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.600157:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35896:x1875453360296704:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 111us (480us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.600165:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3916 00000100:00100000:1.0:1788572540.600312:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 998 00000100:00100000:1.0:1788572540.600322:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360296960:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.600333:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572540.600353:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef55b expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572540.600383:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x43:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737c, blocks: 0x1 00010000:00010000:1.0:1788572540.600402:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=888s v=(0 0) 00010000:00010000:1.0:1788572540.600407:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.600417:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.600435:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.600448:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715800, rc 0) 00000100:00100000:1.0:1788572540.600528:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360296960:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 208us (791us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.600534:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 998 00000100:00100000:1.0:1788572540.600551:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 999 00000100:00100000:1.0:1788572540.600558:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360296832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.600565:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572540.600575:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef554 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572540.600587:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x44:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737c, blocks: 0x1 00010000:00010000:1.0:1788572540.600596:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=888s v=(0 0) 00010000:00010000:1.0:1788572540.600600:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.600605:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.600616:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.600631:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3714000, rc 0) 00000100:00100000:1.0:1788572540.600649:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360296832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 92us (838us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.600656:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 999 00000100:00100000:0.0:1788572540.604733:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.604761:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.604802:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360297088 00000100:00100000:0.0:1788572540.604812:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.604812:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360297088 00000100:00100000:3.0:1788572540.605033:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360297344 00000100:00100000:3.0:1788572540.605037:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360297344 00000100:00100000:2.0:1788572540.605084:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360297600 00000100:00100000:2.0:1788572540.605089:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360297600 00000100:00100000:2.0:1788572540.605119:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3917 00000100:00100000:2.0:1788572540.605129:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35896:x1875453360297088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.605152:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.605175:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161c68 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef562 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.605262:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161c6f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.605276:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161c6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.605287:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161c6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.605322:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.605341:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161c6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.605368:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161c68 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef562 expref: 104 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.605385:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161c6f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef562 expref: 103 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.605394:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:2.0:1788572540.605397:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572540.605401:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53000, rc 0) 00000100:00100000:2.0:1788572540.605454:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35896:x1875453360297088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 326us (729us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.605473:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3917 00000100:00100000:2.0:1788572540.605492:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3918 00000100:00100000:2.0:1788572540.605499:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35895:x1875453360297344:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.605543:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35895:x1875453360297344:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 45us (780us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.605550:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3918 00000100:00100000:2.0:1788572540.605562:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3919 00000100:00100000:2.0:1788572540.605569:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35894:x1875453360297600:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.605602:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35894:x1875453360297600:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 34us (800us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.605609:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3919 00000100:00100000:3.0:1788572540.612195:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.612389:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.612576:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360297728 00000100:00100000:0.0:1788572540.612586:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360297728 00000100:00100000:1.0:1788572540.612631:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.612949:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360297856 00000100:00100000:0.0:1788572540.612954:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360297856 00000100:00100000:0.0:1788572540.612986:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360297984 00000100:00100000:0.0:1788572540.612989:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360297984 00000100:00100000:0.0:1788572540.613031:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1000 00000100:00100000:0.0:1788572540.613041:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35895:x1875453360297728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.613062:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.613081:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef569 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.613112:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 16 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.613121:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572540.613125:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572540.613131:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 16 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.613168:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 16 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.613183:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 16 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.613200:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382340864:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.613443:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1001 00000100:00100000:1.0:1788572540.613453:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360297856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.613472:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572540.613487:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef570 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572540.613511:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x44:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737c, blocks: 0x1 00010000:00010000:1.0:1788572540.613524:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=888s v=(0 0) 00010000:00010000:1.0:1788572540.613527:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.613535:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.613551:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.613569:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679800, rc 0) 00000100:00100000:1.0:1788572540.613603:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360297856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 152us (1214us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.613613:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1001 00000100:00100000:1.0:1788572540.613632:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1002 00000100:00100000:0.0:1788572540.613633:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca973840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.613638:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35894:x1875453360297984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.613645:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572540.613656:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef577 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.613674:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 17 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.613683:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.613685:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.613689:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.613709:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 17 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.613718:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 17 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.613728:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382340992:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.613758:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3780 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.619635:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382340864:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.619679:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.619709:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad400, rc 0) 00000100:00100000:1.0:1788572540.619772:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35895:x1875453360297728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6731us (7586us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.619781:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1000 00000100:00100000:0.0:1788572540.620326:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.620363:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.620400:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.620564:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360298496 00000100:00100000:3.0:1788572540.620571:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360298496 00010000:00010000:3.0:1788572540.620607:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 17 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.620641:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360298368 00000100:00100000:3.0:1788572540.620644:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360298368 00010000:00010000:3.0:1788572540.620655:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 18 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.620682:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 928 00000100:00100000:3.0:1788572540.620691:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360298496:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.620705:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.620730:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 18 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.620742:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 18 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.620768:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.620772:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.620780:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 18 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.620791:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 18 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.620801:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 929 00000100:00100000:1.0:1788572540.620810:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360298368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.620819:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.620829:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 18 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.620837:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 18 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.620841:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 18 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.620855:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382341120:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572540.620865:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000280c545/0x85ace67feb161c5a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef55b expref: 18 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.620877:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.620880:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.620884:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.620892:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.620914:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572540.620915:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.620924:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382341248:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572540.620934:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360298240 00000100:00100000:3.0:1788572540.620938:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360298240 00000100:00100000:1.0:1788572540.620960:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57a0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.620967:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3920 00000100:00100000:3.0:1788572540.620974:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35896:x1875453360298240:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.621069:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35896:x1875453360298240:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 95us (744us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.621077:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3920 00010000:00010000:2.0:1788572540.622733:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161c61 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef554 expref: 17 pid: 13272 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.622754:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382340992:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.622784:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.622800:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:2.0:1788572540.624542:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35894:x1875453360297984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 10891us (11900us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.624554:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1002 00000100:00100000:0.0:1788572540.629516:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.629622:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382341120:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572540.629622:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382341248:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572540.629647:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572540.629647:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572540.629697:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360298368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8888us (9296us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.629712:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360298624 00000100:00100000:2.0:1788572540.629713:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 929 00000100:00100000:3.0:1788572540.629720:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360298624 00002000:00100000:3.0:1788572540.629764:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x43:0x0] (0->2) req@ffff9627dfdbc000 x1875453360298624/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/0 e 0 to 0 dl 1788572551 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572540.629775:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbc000 with x1875453360298624 ext(0->2) 00002000:00010000:3.0:1788572540.629780:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbc000 00000100:00100000:3.0:1788572540.629815:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 229 00000100:00100000:3.0:1788572540.629823:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360298624:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.642454:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360298496:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 21747us (22073us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.642481:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 928 00000100:00100000:0.0:1788572540.646549:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.646703:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360298752 00000100:00100000:1.0:1788572540.646713:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360298752 00000100:00100000:2.0:1788572540.646840:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1003 00000100:00100000:2.0:1788572540.646853:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35896:x1875453360298752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.646873:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.646892:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef57e expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.646925:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.646934:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.646943:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572540.646949:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.646985:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.646999:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 14 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.647031:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382341376:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.647106:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d300 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.650795:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360298624:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 20961us (21277us total) trans 4294967553 rc 0/0 00000100:00100000:2.0:1788572540.650810:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 229 00002000:00100000:2.0:1788572540.650823:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x43:0x0] (0->2) req@ffff9627dfdbc000 x1875453360298624/t4294967553(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/448 e 0 to 0 dl 1788572551 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572540.650834:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbc000 with x1875453360298624 ext(0->2) 00002000:00010000:2.0:1788572540.650841:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbc000 00000100:00100000:0.0:1788572540.651871:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.652041:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360298880 00000100:00100000:2.0:1788572540.652050:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360298880 00002000:00100000:2.0:1788572540.652097:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x44:0x0] (0->2) req@ffff9627c9a5b100 x1875453360298880/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/0 e 0 to 0 dl 1788572551 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572540.652107:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a5b100 with x1875453360298880 ext(0->2) 00002000:00010000:2.0:1788572540.652111:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a5b100 00000100:00100000:2.0:1788572540.652138:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 230 00000100:00100000:2.0:1788572540.652147:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360298880:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.652526:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360298880:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 379us (659us total) trans 4294967554 rc 0/0 00000100:00100000:2.0:1788572540.652535:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 230 00002000:00100000:2.0:1788572540.652542:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x44:0x0] (0->2) req@ffff9627c9a5b100 x1875453360298880/t4294967554(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/448 e 0 to 0 dl 1788572551 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572540.652550:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a5b100 with x1875453360298880 ext(0->2) 00002000:00010000:2.0:1788572540.652554:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a5b100 00000100:00100000:0.0:1788572540.668192:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.668565:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.668683:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299008 00000100:00100000:1.0:1788572540.668693:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299008 00000100:00100000:1.0:1788572540.668760:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3921 00000100:00100000:1.0:1788572540.668769:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35895:x1875453360299008:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.669149:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299136 00000100:00100000:3.0:1788572540.669154:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299136 00010000:00010000:3.0:1788572540.669180:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 15 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.669214:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 930 00000100:00100000:3.0:1788572540.669221:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a59880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360299136:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572540.669234:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572540.669255:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 15 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.669275:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 15 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.669307:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.669312:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.669321:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.669331:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.669370:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.669385:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382341504:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572540.669481:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194cc0 going to sleep for 11 seconds 00010000:00010000:2.0:1788572540.669824:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb161c7d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef570 expref: 15 pid: 6682 timeout: 988 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.669840:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382341376:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.669865:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.669875:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:2.0:1788572540.669915:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35896:x1875453360298752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 23065us (23376us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.669923:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1003 00000100:00100000:0.0:1788572540.679793:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.681352:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299264 00000100:00100000:3.0:1788572540.681365:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299264 00000100:00100000:3.0:1788572540.681421:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3922 00000100:00100000:3.0:1788572540.681431:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35894:x1875453360299264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.681578:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35894:x1875453360299264:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 148us (1794us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.681585:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3922 00000100:00100000:3.0:1788572540.681641:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382341504:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572540.681665:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572540.681685:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a59880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360299136:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 12465us (13384us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.681693:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 930 00000100:00100000:1.0:1788572540.688348:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.688430:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.688526:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299392 00000100:00100000:2.0:1788572540.688536:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299392 00002000:00100000:2.0:1788572540.688589:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x44:0x0] (0->2) req@ffff9626c36c4000 x1875453360299392/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/0 e 0 to 0 dl 1788572551 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572540.688600:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c4000 with x1875453360299392 ext(0->2) 00002000:00010000:2.0:1788572540.688606:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36c4000 00000100:00100000:2.0:1788572540.688636:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 231 00000100:00100000:2.0:1788572540.688645:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360299392:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.689120:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360299392:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 473us (780us total) trans 4294967555 rc 0/0 00000100:00100000:2.0:1788572540.689128:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 231 00002000:00100000:2.0:1788572540.689136:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x44:0x0] (0->2) req@ffff9626c36c4000 x1875453360299392/t4294967555(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:201/0 lens 496/448 e 0 to 0 dl 1788572551 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572540.689144:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c4000 with x1875453360299392 ext(0->2) 00002000:00010000:2.0:1788572540.689148:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36c4000 00000100:00100000:2.0:1788572540.689174:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299520 00000100:00100000:2.0:1788572540.689178:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299520 00000100:00100000:2.0:1788572540.689208:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1004 00000100:00100000:2.0:1788572540.689215:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360299520:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.707914:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.708125:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299648 00000100:00100000:3.0:1788572540.708136:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299648 00000100:00100000:3.0:1788572540.708190:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3923 00000100:00100000:3.0:1788572540.708199:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35896:x1875453360299648:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572540.708350:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35896:x1875453360299648:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 151us (445us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.708357:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3923 00000100:00100000:0.0:1788572540.714253:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.714889:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299776 00000100:00100000:3.0:1788572540.714900:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299776 00000100:00100000:3.0:1788572540.714955:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1005 00000100:00100000:3.0:1788572540.714965:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360299776:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1788572540.747227:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970241 is committed 00000004:00080000:2.0:1788572540.747243:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 34 committed 00000100:00100000:3.0:1788572540.747277:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35895:x1875453360299008:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 78503us (79089us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.747289:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3921 00000020:00080000:2.0:1788572540.747610:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c9326b40 refcount 2 committed 0 result 0 batchid 97 00000020:00080000:2.0:1788572540.747615:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae700 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c9326f00 00000020:00080000:2.0:1788572540.747618:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aef00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffd0cc00 00000020:00080000:2.0:1788572540.747640:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 96 batchid updated 00000020:00080000:2.0:1788572540.747657:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2eb50c0 refcount 2 committed 0 result 0 batchid 99 00000020:00080000:2.0:1788572540.747660:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627d1608f00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2eb56c0 00000020:00080000:2.0:1788572540.747662:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627d1608000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627dffe6000 00000020:00080000:2.0:1788572540.747669:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c9bd3900 refcount 2 committed 0 result 0 batchid 98 00000020:00080000:2.0:1788572540.747671:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627d1608300 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c9bd3c00 00000020:00080000:2.0:1788572540.747674:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627d1608c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f56f6400 00000004:00080000:2.0:1788572540.747686:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 39 committed 00000100:00100000:1.0:1788572540.748153:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382341760:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572540.748214:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:0.0:1788572540.748243:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 99 committed batchid 96 00000100:00100000:1.0:1788572540.748259:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2734000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382342272:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572540.748281:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572540.748402:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382341888:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572540.748429:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572540.748455:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382342016:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572540.748499:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572540.748528:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382341760 00000100:00100000:3.0:1788572540.748535:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382341760 00000100:00100000:3.0:1788572540.748571:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382342272 00000100:00100000:3.0:1788572540.748575:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382342272 00000100:00100000:3.0:1788572540.748586:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382341888 00000100:00100000:3.0:1788572540.748588:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382341888 00000100:00100000:3.0:1788572540.748595:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382342016 00000100:00100000:3.0:1788572540.748597:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382342016 00000100:00100000:3.0:1788572540.748609:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1006 00000100:00100000:3.0:1788572540.748616:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3303:x1875453382341760:12345-0@lo:2:osp-syn-1-0.0 00000020:00080000:0.0:1788572540.749396:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 96 00000100:00100000:1.0:1788572540.749409:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572540.749421:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382341632:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572540.749456:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572540.749485:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382342144:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572540.749508:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572540.749633:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382341632 00000100:00100000:1.0:1788572540.749639:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382341632 00000100:00100000:1.0:1788572540.749665:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382342144 00000100:00100000:1.0:1788572540.749668:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382342144 00000100:00100000:1.0:1788572540.749680:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1007 00000100:00100000:1.0:1788572540.749687:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382342272:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1788572540.749928:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572540.749933:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb mtime from disk: 0 -> 1788572540 00002000:00010000:1.0:1788572540.749935:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572540.749938:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb ctime from disk: 1788572540 -> 1788572540 00002000:00010000:1.0:1788572540.749940:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572540.749980:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382342272:12345-0@lo:2:osp-syn-0-0.0 Request processed in 293us (1698us total) trans 4294967556 rc 0/0 00000100:00100000:1.0:1788572540.749988:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1007 00000100:00100000:1.0:1788572540.750022:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1008 00000100:00100000:1.0:1788572540.750029:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382341888:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1788572540.750111:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572540.750115:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb mtime from disk: 0 -> 1788572540 00002000:00010000:1.0:1788572540.750117:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572540.750119:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb ctime from disk: 1788572540 -> 1788572540 00002000:00010000:1.0:1788572540.750121:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572540.750148:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382341888:12345-0@lo:2:osp-syn-1-0.0 Request processed in 120us (1718us total) trans 4294967556 rc 0/0 00000100:00100000:1.0:1788572540.750154:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1008 00000100:00100000:1.0:1788572540.750167:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1009 00000100:00100000:1.0:1788572540.750173:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382342016:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572540.750271:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382342016:12345-0@lo:2:osp-syn-0-0.0 Request processed in 98us (1770us total) trans 4294967557 rc 0/0 00000100:00100000:1.0:1788572540.750277:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1009 00000100:00100000:1.0:1788572540.750291:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1010 00000100:00100000:1.0:1788572540.750296:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382341632:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572540.750880:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382341760:12345-0@lo:2:osp-syn-1-0.0 Request processed in 2261us (2663us total) trans 4294967555 rc 0/0 00000100:00100000:3.0:1788572540.750887:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1006 00000100:00100000:3.0:1788572540.750904:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1011 00000100:00100000:3.0:1788572540.750911:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382342144:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1788572540.751030:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382342144:12345-0@lo:2:osp-syn-0-0.0 Request processed in 120us (1521us total) trans 4294967558 rc 0/0 00000100:00100000:3.0:1788572540.751036:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1011 00000004:00080000:3.0:1788572540.751091:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627fb472d80/2, rc 0, transno 4294967556 00000100:00100000:3.0:1788572540.751105:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382341888:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572540.751136:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e786ea00/2, rc 0, transno 4294967557 00000100:00100000:3.0:1788572540.751141:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382342016:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1788572540.751718:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e786ed80/2, rc 0, transno 4294967558 00000100:00100000:0.0:1788572540.751725:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382342144:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1788572540.751733:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627fb472680/2, rc 0, transno 4294967555 00000100:00100000:0.0:1788572540.751740:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382341760:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1788572540.751748:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9626c2734000/2, rc 0, transno 4294967556 00000100:00100000:0.0:1788572540.751751:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2734000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382342272:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572540.752158:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382341632:12345-0@lo:2:osp-syn-1-0.0 Request processed in 1860us (2700us total) trans 4294967557 rc 0/0 00000100:00100000:1.0:1788572540.752166:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1010 00000004:00080000:0.0:1788572540.752209:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff962801f94700/2, rc 0, transno 4294967557 00000100:00100000:0.0:1788572540.752540:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382341632:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572540.755916:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.755948:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360299904 00000100:00100000:3.0:1788572540.755957:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360299904 00000100:00100000:3.0:1788572540.756028:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1012 00000100:00100000:3.0:1788572540.756040:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360299904:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572540.769998:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360299520:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 80769us (81553us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.770040:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1004 00000001:00080000:3.0:1788572540.770752:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967549 is committed 00000001:00080000:3.0:1788572540.770764:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967551 is committed 00000001:00080000:3.0:1788572540.770770:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967554 is committed 00000001:00080000:3.0:1788572540.770783:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967550 is committed 00000100:00100000:1.0:1788572540.788930:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.789020:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360300160 00000100:00100000:0.0:1788572540.789032:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360300160 00000100:00100000:0.0:1788572540.789093:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3924 00000100:00100000:0.0:1788572540.789103:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35894:x1875453360300160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.789660:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.789690:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb161c53 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef54d expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.789711:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:0.0:1788572540.789716:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.789746:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.789759:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.789767:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572540.789913:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b760cf00 reply state 000000001c5e7d87 transno 4294970242 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.789922:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.790070:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35894:x1875453360300160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 967us (1148us total) trans 4294970242 rc 0/0 00000100:00100000:0.0:1788572540.790077:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3924 00010000:00010000:1.0:1788572540.793086:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.793168:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.793178:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.793184:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.793188:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.793196:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.793212:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572540.793215:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.793220:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.793230:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161c92 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.794873:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.794956:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360300288 00000100:00100000:1.0:1788572540.794966:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360300288 00000100:00100000:1.0:1788572540.795039:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1013 00000100:00100000:1.0:1788572540.795049:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360300288:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572540.795448:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:1.0:1788572540.795452:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb mtime from disk: 1788572540 -> 1788572516 00002000:00010000:1.0:1788572540.795455:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:1.0:1788572540.795457:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb ctime from disk: 1788572540 -> 1788572527 00002000:00010000:1.0:1788572540.795459:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x44:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:1.0:1788572540.795555:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360300288:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 505us (691us total) trans 4294967558 rc 0/0 00000100:00100000:1.0:1788572540.795564:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1013 00000001:00080000:0.0:1788572540.799370:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967553 is committed 00000001:00080000:0.0:1788572540.799398:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967555 is committed 00000100:00100000:0.0:1788572540.799570:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360299776:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 84599us (85320us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.799586:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1005 00000100:00100000:2.0:1788572540.804304:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.804343:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.804440:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360300544 00000100:00100000:0.0:1788572540.804451:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360300544 00000100:00100000:0.0:1788572540.804511:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360300800 00000100:00100000:0.0:1788572540.804515:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360300800 00000100:00100000:0.0:1788572540.804536:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3925 00000100:00100000:0.0:1788572540.804546:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35894:x1875453360300544:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.804619:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.804640:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.804650:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.804688:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.804702:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.804718:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb471500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382342400:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572540.804808:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51480 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.804828:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3926 00000100:00100000:0.0:1788572540.804837:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35896:x1875453360300800:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.804872:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.804892:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161c6f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef562 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.804908:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:0.0:1788572540.804913:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.805030:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.805044:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.805054:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572540.805297:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3492a29 reply state 0000000082c8347f transno 4294970243 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.805307:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.805379:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35896:x1875453360300800:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 542us (1033us total) trans 4294970243 rc 0/0 00000100:00100000:0.0:1788572540.805387:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3926 00000001:00080000:3.0:1788572540.813318:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967557 is committed 00000100:00100000:1.0:1788572540.813488:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360299904:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 57445us (57577us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.813500:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1012 00000100:00100000:3.0:1788572540.814580:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb471500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382342400:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.814607:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.820346:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.820423:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360300928 00000100:00100000:3.0:1788572540.820431:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360300928 00000100:00100000:3.0:1788572540.820503:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1014 00000100:00100000:3.0:1788572540.820512:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360300928:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572540.820711:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572540.820715:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb mtime from disk: 1788572540 -> 1788572515 00002000:00010000:3.0:1788572540.820717:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb atime from disk: 0 -> 1788572515 00002000:00010000:3.0:1788572540.820719:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb ctime from disk: 1788572540 -> 1788572527 00002000:00010000:3.0:1788572540.820720:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x44:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572540.820785:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360300928:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 274us (445us total) trans 4294967559 rc 0/0 00000100:00100000:3.0:1788572540.820792:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1014 00010000:00010000:1.0:1788572540.821127:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.821136:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.821143:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.821153:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.821157:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.821165:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.821179:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:1.0:1788572540.821181:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.821186:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.821191:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161ca0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.821321:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.821382:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360301056 00000100:00100000:0.0:1788572540.821387:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360301056 00010000:00010000:0.0:1788572540.821411:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.821440:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 931 00000100:00100000:0.0:1788572540.821447:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360301056:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572540.821459:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.821475:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.821485:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.821498:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161bd5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef51c expref: 66 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.821507:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:0.0:1788572540.821511:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.821519:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.821527:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.821552:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.821585:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360301056:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 139us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.821591:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 931 00010000:00010000:3.0:1788572540.821748:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821755:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821782:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821792:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821798:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821802:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572540.821805:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821811:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821819:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:3.0:1788572540.821821:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572540.821824:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.821829:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161c99 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.821875:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35894:x1875453360300544:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 17330us (17580us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572540.821883:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3925 00000100:00100000:2.0:1788572540.841703:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.841753:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.841786:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.844707:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360301312 00000100:00100000:0.0:1788572540.844723:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360301312 00000100:00100000:0.0:1788572540.844797:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360301568 00000100:00100000:0.0:1788572540.844801:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360301568 00000100:00100000:0.0:1788572540.844830:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3927 00000100:00100000:0.0:1788572540.844841:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35895:x1875453360301312:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.844899:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.844926:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161c4c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef546 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.844960:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:0.0:1788572540.844965:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572540.844998:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.845056:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.845065:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572540.845226:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f4131235 reply state 00000000790ca0c9 transno 4294970244 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.845234:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572540.845353:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35895:x1875453360301312:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 513us (3660us total) trans 4294970244 rc 0/0 00000100:00100000:0.0:1788572540.845362:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3927 00000100:00100000:0.0:1788572540.845380:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3928 00000100:00100000:0.0:1788572540.845387:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35896:x1875453360301568:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.845417:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.845432:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.845440:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 4/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.845485:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.845499:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.845514:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb471500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382342528:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572540.845565:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.845839:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360301696 00000100:00100000:1.0:1788572540.845846:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360301696 00000100:00100000:1.0:1788572540.845889:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1015 00000100:00100000:1.0:1788572540.845897:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360301696:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.845922:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.846080:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.846088:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.846092:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.846096:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.846121:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.846131:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 12 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.846143:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4921c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382342656:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.846174:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.850651:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4921c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382342656:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.850684:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.850777:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb471500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382342528:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.850797:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851115:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851124:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851131:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851139:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.851144:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851152:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851168:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:1.0:1788572540.851171:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572540.851175:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.851185:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161ca7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.853471:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.853518:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.853556:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.853627:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360301824 00000100:00100000:1.0:1788572540.853636:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360301824 00010000:00010000:1.0:1788572540.853675:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.853713:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360301952 00000100:00100000:1.0:1788572540.853717:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360301952 00010000:00010000:1.0:1788572540.853732:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 13 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.853755:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 932 00000100:00100000:1.0:1788572540.853765:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360301824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.853780:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.853787:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.853800:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.853823:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161c22 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x79:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef531 expref: 65 pid: 22536 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.853839:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:1.0:1788572540.853844:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.853855:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.853864:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.853891:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.853950:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360301824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 187us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.853958:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 932 00000100:00100000:1.0:1788572540.853975:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 933 00000100:00100000:1.0:1788572540.853981:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e51450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360301952:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.853990:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.854037:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 13 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854048:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 13 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854061:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb161c84 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef577 expref: 13 pid: 6682 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854074:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.854077:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.854083:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854097:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854118:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.854139:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e51450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360301952:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 158us (621us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.854145:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 933 00010000:00010000:1.0:1788572540.854197:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854206:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854231:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854238:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854246:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854253:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572540.854256:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854270:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854281:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854288:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c59938e/0x85ace67feb161cb5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.854329:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360301696:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 8433us (12541us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.854337:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1015 00000100:00100000:1.0:1788572540.854355:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360302080 00000100:00100000:1.0:1788572540.854359:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360302080 00000100:00100000:1.0:1788572540.854384:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1016 00000100:00100000:1.0:1788572540.854391:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360302080:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572540.854591:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572540.854595:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb mtime from disk: 0 -> 1788572515 00002000:00010000:1.0:1788572540.854598:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb atime from disk: 0 -> 1788572515 00002000:00010000:1.0:1788572540.854600:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb ctime from disk: 1788572540 -> 1788572527 00002000:00010000:1.0:1788572540.854602:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572540.854624:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360302080:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 233us (1067us total) trans 4294967559 rc 0/0 00000100:00100000:1.0:1788572540.854631:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1016 00010000:00010000:1.0:1788572540.854765:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854772:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854798:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854804:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854810:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854815:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.854819:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854826:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854836:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:1.0:1788572540.854839:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572540.854842:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 2/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.854849:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161cae lrc: 0/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.854896:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35896:x1875453360301568:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 9508us (13142us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572540.854908:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3928 00000100:00100000:2.0:1788572540.872228:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.872274:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.872309:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.872402:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360302464 00000100:00100000:1.0:1788572540.872413:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360302464 00000100:00100000:1.0:1788572540.872483:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360302720 00000100:00100000:1.0:1788572540.872487:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360302720 00000100:00100000:1.0:1788572540.872515:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3929 00000100:00100000:1.0:1788572540.872527:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35894:x1875453360302464:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.872619:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872637:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872649:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872883:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872896:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872905:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872947:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.872962:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 65 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.872978:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4922a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382342784:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.873090:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788acc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572540.873117:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3930 00000100:00100000:1.0:1788572540.873126:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35895:x1875453360302720:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.873157:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.873168:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.873175:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 4/0,1 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.873192:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.873201:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 65 pid: 9040 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.873210:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4921f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382342912:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.873233:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.873708:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360302208 00000100:00100000:3.0:1788572540.873717:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360302208 00000100:00100000:3.0:1788572540.873763:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1017 00000100:00100000:3.0:1788572540.873772:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360302208:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572540.873803:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.873820:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 11 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.873827:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572540.873832:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572540.873836:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.873859:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 11 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.873869:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 11 pid: 6681 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.873879:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ebb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382343040:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572540.873926:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3251480 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.879367:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.879418:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.879446:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360302976 00000100:00100000:2.0:1788572540.879455:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360302976 00010000:00010000:2.0:1788572540.879507:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 66 pid: 9040 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.879541:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360303104 00000100:00100000:2.0:1788572540.879544:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360303104 00010000:00010000:2.0:1788572540.879560:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 12 pid: 6681 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.879582:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 934 00000100:00100000:2.0:1788572540.879591:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360302976:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572540.879608:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572540.879616:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 66 pid: 9040 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879628:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 66 pid: 9040 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879651:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:2.0:1788572540.879656:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879666:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879675:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879695:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572540.879741:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360302976:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 151us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.879749:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 934 00000100:00100000:2.0:1788572540.879768:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 935 00000100:00100000:2.0:1788572540.879775:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360303104:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572540.879783:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572540.879807:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 12 pid: 6681 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879818:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 12 pid: 6681 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879835:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572540.879840:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572540.879845:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879854:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.879867:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572540.879885:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360303104:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 111us (467us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.879891:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 935 00010000:00010000:0.0:1788572540.880046:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb161c8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x44:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef57e expref: 11 pid: 6681 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.880062:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ebb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382343040:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572540.880082:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880094:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880102:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880133:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880141:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880148:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880155:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572540.880160:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880169:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880181:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.880188:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077cf3010/0x85ace67feb161cd1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.880233:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360302208:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 6463us (8016us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.880241:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1017 00000100:00100000:1.0:1788572540.880456:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4922a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382342784:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.880508:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880595:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161bf1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x78:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef523 expref: 65 pid: 9040 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.880608:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4921f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382342912:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572540.880622:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880629:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880635:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880659:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880665:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880670:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880676:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e76ea200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572540.880680:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880686:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880695:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:3.0:1788572540.880698:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572540.880701:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 2/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.880706:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb161cca lrc: 0/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572540.880760:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35895:x1875453360302720:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 7636us (8450us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572540.880768:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3930 00000100:00100000:2.0:1788572540.881502:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.881571:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360302848 00000100:00100000:1.0:1788572540.881579:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360302848 00010000:00010000:1.0:1788572540.881605:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 64 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.881637:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 936 00000100:00100000:1.0:1788572540.881645:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2348:x1875453360302848:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.881657:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.881662:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 64 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.881673:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 64 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.881689:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb161c14 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef52a expref: 64 pid: 5838 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.881703:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572540.881707:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.881716:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.881724:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.881812:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.881847:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2348:x1875453360302848:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 204us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.881855:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 936 00000100:00100000:1.0:1788572540.886572:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.886671:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360303360 00000100:00100000:2.0:1788572540.886681:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360303360 00000100:00100000:2.0:1788572540.886742:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3931 00000100:00100000:2.0:1788572540.886752:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35896:x1875453360303360:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.886839:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.886856:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.886867:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.887150:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.887165:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 64 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.887173:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 4/0,1 mode: --/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.887207:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 64 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.887235:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 64 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.887248:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382343168:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572540.887335:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e77349c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.889992:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.890139:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382343168:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572540.890167:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.893442:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.893472:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.893710:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360303488 00000100:00100000:1.0:1788572540.893722:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360303488 00000100:00100000:1.0:1788572540.893807:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1018 00000100:00100000:1.0:1788572540.893818:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360303488:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.893861:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.893881:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 12 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.893890:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572540.893894:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.893898:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.893936:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 12 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.893949:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 12 pid: 10416 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.893963:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382343296:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572540.894075:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000001:00080000:2.0:1788572540.895404:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970245 00000001:00080000:2.0:1788572540.895598:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970247 00000001:00080000:2.0:1788572540.895647:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970248 00000100:00100000:2.0:1788572540.895734:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382343424:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572540.895795:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572540.895831:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3fee40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.895961:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382343424 00000100:00100000:3.0:1788572540.895969:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382343424 00000100:00100000:3.0:1788572540.896037:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00000100:00100000:3.0:1788572540.896046:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382343424:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572540.896121:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382343424:12345-0@lo:1000:mdt00_004.0 Request processed in 76us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.896128:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:2.0:1788572540.896914:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967557 gen 2 00000004:00080000:2.0:1788572540.896922:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967550 committed transno 4294967557 00000004:00080000:2.0:1788572540.896955:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627fb472680, transno 4294967555 00000004:00080000:2.0:1788572540.896971:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627fb472d80, transno 4294967556 00000004:00080000:2.0:1788572540.896974:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff962801f94700, transno 4294967557 00000100:00100000:2.0:1788572540.896983:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382343424:0@lo:1000:mdt00_004.0 00000001:00080000:2.0:1788572540.897382:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970249 00000100:00100000:2.0:1788572540.897453:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382343552:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572540.897509:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572540.897546:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382343552 00000100:00100000:2.0:1788572540.897551:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382343552 00000100:00100000:2.0:1788572540.897589:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1788572540.897597:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382343552:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572540.897772:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382343552:12345-0@lo:1000:mdt00_004.0 Request processed in 176us (262us total) trans 4294967560 rc 0/0 00000100:00100000:2.0:1788572540.897780:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1788572540.897813:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2eb5000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572540.897836:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382343552:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1788572540.897868:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897878:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897886:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897892:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.897896:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897908:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897925:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:2.0:1788572540.897928:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.897934:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897939:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb161cc3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x7a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897972:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897978:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897984:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897989:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.897992:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.897998:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898028:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572540.898030:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.898033:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898042:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb15edde lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898065:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898077:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898086:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898094:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898099:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898105:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898110:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.898113:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898119:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898126:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572540.898128:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.898131:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898137:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161ced lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898146:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898153:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898158:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898163:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.898165:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898170:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898177:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572540.898179:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572540.898182:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898188:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb161cbc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbb:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.898334:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35894:x1875453360302464:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 25807us (26056us total) trans 4294970250 rc 0/0 00000100:00100000:2.0:1788572540.898344:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3929 00000020:00080000:2.0:1788572540.898349:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:2.0:1788572540.898401:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.898528:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360303616 00000100:00100000:2.0:1788572540.898533:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360303616 00010000:00010000:2.0:1788572540.898556:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 68 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.898582:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 937 00000100:00100000:2.0:1788572540.898590:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360303616:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572540.898602:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572540.898609:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 68 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898621:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 68 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898637:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161c37 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef53f expref: 68 pid: 23058 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898646:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:2.0:1788572540.898650:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898658:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898665:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.898683:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572540.898708:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360303616:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 119us (306us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.898714:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 937 00000004:00080000:1.0:1788572540.900830:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:1.0:1788572540.901132:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382343296:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572540.901156:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.902409:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.902423:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572540.902611:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970246 00000001:00080000:0.0:1788572540.902767:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970251 00000001:00080000:0.0:1788572540.902817:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970252 00000100:00100000:0.0:1788572540.902919:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382343680:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572540.902996:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572540.903062:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8180 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.903134:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382343680 00000100:00100000:3.0:1788572540.903141:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382343680 00000100:00100000:3.0:1788572540.903195:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:3.0:1788572540.903205:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:23058:x1875453382343680:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572540.903281:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:23058:x1875453382343680:12345-0@lo:1000:mdt00_005.0 Request processed in 78us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.903289:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:0.0:1788572540.903362:0:23058:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967553 gen 2 00000004:00080000:0.0:1788572540.903368:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967552 committed transno 4294967553 00000004:00080000:0.0:1788572540.903384:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967553 committed transno 4294967553 00000100:00100000:0.0:1788572540.903400:0:23058:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9626c2734000 x1875453382342272/t4294967556(4294967556) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1788572556 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572540.903411:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382343680:0@lo:1000:mdt00_005.0 00000001:00080000:0.0:1788572540.903736:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970253 00000100:00100000:0.0:1788572540.903789:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382343808:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572540.903817:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572540.903843:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b86c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.903988:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382343808 00000100:00100000:3.0:1788572540.903993:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382343808 00000100:00100000:3.0:1788572540.904049:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1788572540.904056:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:23058:x1875453382343808:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572540.904206:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:23058:x1875453382343808:12345-0@lo:1000:mdt00_005.0 Request processed in 149us (388us total) trans 4294967560 rc 0/0 00000100:00100000:3.0:1788572540.904213:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1788572540.904596:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.904677:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.904875:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360303744 00000100:00100000:1.0:1788572540.904880:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360303744 00010000:00010000:1.0:1788572540.904907:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 13 pid: 10416 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.904951:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 938 00000100:00100000:1.0:1788572540.904959:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360303744:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572540.904971:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572540.904992:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 13 pid: 10416 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905033:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 13 pid: 10416 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905055:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dcd4dc8/0x85ace67feb161c76 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x43:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef569 expref: 13 pid: 10416 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905070:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.905074:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572540.905080:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905092:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905117:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572540.905156:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360303744:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 198us (562us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.905163:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 938 00010000:00010000:1.0:1788572540.905184:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905193:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905213:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905221:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905229:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905237:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572540.905242:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905258:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905269:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.905276:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e2e58ba0/0x85ace67feb161ce6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.905319:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360303488:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11504us (15337us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.905329:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1018 00000100:00100000:2.0:1788572540.906093:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360304000 00000100:00100000:2.0:1788572540.906101:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360304000 00000100:00100000:2.0:1788572540.906140:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 863 00000100:00100000:2.0:1788572540.906148:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35894:x1875453360304000:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572540.906769:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x7a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572540.906881:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970255 00010000:00010000:2.0:1788572540.906930:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.906958:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.906969:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.906977:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.906983:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.906988:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.907142:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382343808:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1788572540.907178:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907186:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907197:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907203:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.907207:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907214:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907226:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627e7451300) 00010000:00010000:0.0:1788572540.907229:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572540.907233:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 2/0,0 mode: --/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907240:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52c00/0x85ace67feb161cdf lrc: 0/0,0 mode: --/EX res: [0x200000402:0x79:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907272:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907279:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907285:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907290:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.907293:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907302:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907310:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627d599af00) 00010000:00010000:0.0:1788572540.907312:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572540.907315:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907320:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4400/0x85ace67feb15eee8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907336:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907344:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907351:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907357:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907362:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907367:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907372:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.907374:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907380:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907386:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627d599af00) 00010000:00010000:0.0:1788572540.907388:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.907391:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907396:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161cfb lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907406:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907412:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907418:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907423:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.907425:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907430:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907436:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627d599af00) 00010000:00010000:0.0:1788572540.907438:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572540.907441:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.907449:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161cd8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.907622:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35896:x1875453360303360:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 20869us (21058us total) trans 4294970254 rc 0/0 00000100:00100000:0.0:1788572540.907632:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3931 00000020:00080000:0.0:1788572540.907636:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572540.913977:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.914171:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360304256 00000100:00100000:0.0:1788572540.914182:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360304256 00000100:00100000:0.0:1788572540.914244:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3932 00000100:00100000:0.0:1788572540.914256:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35895:x1875453360304256:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.914342:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914359:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914370:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914614:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914625:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914634:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914674:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.914689:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 74 pid: 5839 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.914703:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382343936:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572540.914792:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d300 going to sleep for 11 seconds 00010000:00010000:2.0:1788572540.915332:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572540.915422:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35894:x1875453360304000:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 9273us (10742us total) trans 4294970256 rc 0/0 00000100:00100000:2.0:1788572540.915435:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 863 00000020:00080000:2.0:1788572540.915440:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572540.916402:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572540.916414:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.916424:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572540.916429:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.916439:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.916456:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7a:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:0.0:1788572540.916459:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572540.916475:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.916481:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161cf4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.916501:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572540.916503:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572540.916506:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572540.918525:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.918650:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360304512 00000100:00100000:3.0:1788572540.918662:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360304512 00000100:00100000:3.0:1788572540.918727:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 864 00000100:00100000:3.0:1788572540.918737:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e65050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35896:x1875453360304512:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572540.919197:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x79:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572540.919241:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970258 00010000:00010000:3.0:1788572540.919288:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.919306:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.919317:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.919327:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.919335:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.919341:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572540.919360:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572540.919456:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e65050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35896:x1875453360304512:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 719us (941us total) trans 4294970259 rc 0/0 00000100:00100000:3.0:1788572540.919477:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 864 00000020:00080000:3.0:1788572540.919481:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572540.919598:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572540.919606:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.919613:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572540.919617:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.919626:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.919639:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x79:0x0].0x0 (ffff9627d599a400) 00010000:00010000:2.0:1788572540.919642:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572540.919647:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.919654:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161d10 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x79:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.919670:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572540.919672:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572540.919674:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572540.919776:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382343936:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572540.919802:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572540.922379:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.922518:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360304640 00000100:00100000:0.0:1788572540.922527:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360304640 00010000:00010000:0.0:1788572540.922562:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 76 pid: 5839 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.922604:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 939 00000100:00100000:0.0:1788572540.922613:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2350:x1875453360304640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572540.922628:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572540.922635:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 76 pid: 5839 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922648:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 76 pid: 5839 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922668:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb161c29 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef538 expref: 76 pid: 5839 timeout: 988 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922685:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:0.0:1788572540.922690:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922700:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922712:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922734:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572540.922777:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2350:x1875453360304640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 165us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.922785:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 939 00010000:00010000:0.0:1788572540.922809:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.922816:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572540.922990:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970257 00000001:00080000:0.0:1788572540.923181:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970260 00000001:00080000:0.0:1788572540.923226:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970261 00000100:00100000:0.0:1788572540.923308:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382344064:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572540.923354:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572540.923381:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9326540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.923401:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382344064 00000100:00100000:0.0:1788572540.923405:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382344064 00000100:00100000:0.0:1788572540.923434:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1788572540.923442:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382344064:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572540.923524:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382344064:12345-0@lo:1000:mdt00_003.0 Request processed in 82us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.923532:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1788572540.923571:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382344064:0@lo:1000:mdt00_003.0 00000001:00080000:0.0:1788572540.923912:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970262 00000100:00100000:0.0:1788572540.923963:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382344192:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572540.923992:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572540.924035:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572540.924051:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382344192 00000100:00100000:0.0:1788572540.924055:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382344192 00000100:00100000:0.0:1788572540.924080:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:0.0:1788572540.924088:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382344192:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572540.927782:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:9040:x1875453382344192:12345-0@lo:1000:mdt00_003.0 Request processed in 3684us (3778us total) trans 4294967561 rc 0/0 00000100:00100000:0.0:1788572540.927798:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1788572540.932277:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382344192:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1788572540.932337:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932348:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932360:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932365:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.932370:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932379:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932397:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7451200) 00010000:00010000:2.0:1788572540.932401:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.932405:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932412:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d09 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x78:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932449:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932456:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932472:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932477:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.932480:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932489:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932498:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:2.0:1788572540.932500:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572540.932503:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932509:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb15ee2b lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932536:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932548:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932558:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932565:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932571:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932577:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932582:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.932585:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932592:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932599:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:2.0:1788572540.932601:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.932603:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932608:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d17 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932618:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932624:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932629:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932634:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.932636:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932642:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932649:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:2.0:1788572540.932651:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572540.932654:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.932664:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d02 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572540.934130:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35895:x1875453360304256:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 19867us (20158us total) trans 4294970263 rc 0/0 00000100:00100000:2.0:1788572540.934143:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3932 00000020:00080000:2.0:1788572540.934149:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:0.0:1788572540.936602:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572540.936612:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572540.936652:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 127 count 101 00000040:00080000:0.0:1788572540.936659:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:0.0:1788572540.936811:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc5180 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382344320:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572540.937048:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788572540.951283:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.951391:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.951418:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305024 00000100:00100000:3.0:1788572540.951426:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305024 00000100:00100000:1.0:1788572540.952030:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc5180 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382344320:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572540.952436:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3933 00000100:00100000:3.0:1788572540.952449:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35897:x1875453360305024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572540.952500:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xc2:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572540.952609:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35897:x1875453360305024:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 162us (1217us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.952617:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3933 00000100:00100000:3.0:1788572540.952665:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.953092:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.953121:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305152 00000100:00100000:3.0:1788572540.953127:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305152 00000100:00100000:3.0:1788572540.953157:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305280 00000100:00100000:3.0:1788572540.953160:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305280 00000100:00100000:3.0:1788572540.953181:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3934 00000100:00100000:3.0:1788572540.953188:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35899:x1875453360305152:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572540.953205:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbd:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572540.953373:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3935 00000100:00100000:1.0:1788572540.953382:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35898:x1875453360305280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572540.953397:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb9:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572540.953475:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35898:x1875453360305280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 83us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572540.953482:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3935 00000100:00100000:3.0:1788572540.953851:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35899:x1875453360305152:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 662us (1183us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572540.953858:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3934 00000100:00100000:1.0:1788572540.957607:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.957683:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.957760:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360304896 00000100:00100000:2.0:1788572540.957770:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360304896 00000100:00100000:2.0:1788572540.957825:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 865 00000100:00100000:2.0:1788572540.957836:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35895:x1875453360304896:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572540.958255:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x78:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572540.958297:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970264 00010000:00010000:2.0:1788572540.958342:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.958359:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.958371:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.958381:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.958387:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.958394:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.958410:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572540.958513:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35895:x1875453360304896:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 677us (7305us total) trans 4294970265 rc 0/0 00000100:00100000:2.0:1788572540.958521:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 865 00000020:00080000:2.0:1788572540.958526:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572540.962304:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572540.962318:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.962328:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572540.962333:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.962344:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.962363:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x78:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:0.0:1788572540.962367:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572540.962374:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 1/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.962381:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161d1e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x78:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.962409:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572540.962411:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572540.962413:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572540.963529:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.963887:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305408 00000100:00100000:3.0:1788572540.963897:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305408 00000100:00100000:3.0:1788572540.963968:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305536 00000100:00100000:3.0:1788572540.963971:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305536 00000100:00100000:0.0:1788572540.964436:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305664 00000100:00100000:0.0:1788572540.964441:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305664 00000100:00100000:0.0:1788572540.964480:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3936 00000100:00100000:0.0:1788572540.964489:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35897:x1875453360305408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.964556:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964569:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964577:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964587:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964591:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964595:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964611:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964617:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964621:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.964655:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc2:0x0]: returning size 3 00000004:00200000:0.0:1788572540.964659:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.964672:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964681:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964688:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964695:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964701:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.964705:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964712:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964722:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.964725:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.964730:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964740:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161d25 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964753:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964758:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964761:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964764:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.964766:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964769:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964773:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xbb5290ce (ffff9627e7451300) 00010000:00010000:0.0:1788572540.964775:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.964776:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964779:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161d2c lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xbb5290ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964790:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964794:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964798:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964801:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.964803:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964806:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964809:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:0.0:1788572540.964810:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.964812:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964815:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161d33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.964881:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35897:x1875453360305408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 393us (7285us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.964886:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3936 00000100:00100000:0.0:1788572540.964899:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3937 00000100:00100000:0.0:1788572540.964903:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35899:x1875453360305536:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.964920:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964925:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964932:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964939:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964943:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964947:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964958:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964962:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.964966:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.964978:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbd:0x0]: returning size 3 00000004:00200000:0.0:1788572540.964980:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.964984:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965001:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965019:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965029:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965034:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.965037:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965042:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965049:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.965052:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.965055:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965060:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb161d3a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965070:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965076:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965080:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965084:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.965085:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965091:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965096:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xd9125233 (ffff9627e7451f00) 00010000:00010000:0.0:1788572540.965097:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.965099:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965103:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb161d41 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd9125233 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965111:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965115:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965118:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965121:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bace00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.965122:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965134:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965141:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627c7d65100) 00010000:00010000:0.0:1788572540.965143:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.965146:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965154:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161d48 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.965184:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35899:x1875453360305536:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 279us (7500us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.965189:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3937 00000100:00100000:0.0:1788572540.965198:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3938 00000100:00100000:0.0:1788572540.965203:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35898:x1875453360305664:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.965220:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965226:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965231:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965440:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965450:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965457:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965486:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965494:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965501:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.965519:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb9:0x0]: returning size 3 00000004:00200000:0.0:1788572540.965522:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572540.965529:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965537:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965542:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965548:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965552:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.965555:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965560:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965567:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.965569:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.965572:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965578:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161d4f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965591:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965596:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965600:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965605:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.965607:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965612:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965617:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3f2d17e0 (ffff9627e7451600) 00010000:00010000:0.0:1788572540.965619:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.965622:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965627:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161d56 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3f2d17e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965638:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965645:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965651:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965656:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.965659:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965666:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965673:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572540.965675:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572540.965679:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.965684:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161d5d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.965719:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35898:x1875453360305664:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 514us (2197us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.965725:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3938 00000100:00100000:1.0:1788572540.969671:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.969781:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360306176 00000100:00100000:0.0:1788572540.969791:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360306176 00000100:00100000:0.0:1788572540.969900:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3939 00000100:00100000:0.0:1788572540.969910:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35898:x1875453360306176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.969934:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.969953:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161d64 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef58c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970101:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970133:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970146:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.970157:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb9:0x0]: returning size 3 00010000:00010000:0.0:1788572540.970257:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970265:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970272:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970277:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.970282:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970290:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970301:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572540.970304:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572540.970308:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970314:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161d6b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970360:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161d64 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef58c expref: 82 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970370:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.970372:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572540.970376:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161d64 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef58c expref: 82 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.970385:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246c000, rc 0) 00000100:00100000:0.0:1788572540.970439:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35898:x1875453360306176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 532us (779us total) trans 4294970266 rc 301/301 00000100:00100000:0.0:1788572540.970447:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3939 00000100:00100000:1.0:1788572540.970541:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.975116:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360305920 00000100:00100000:3.0:1788572540.975131:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360305920 00000100:00100000:0.0:1788572540.976437:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3940 00000100:00100000:0.0:1788572540.976456:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35897:x1875453360305920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572540.976487:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572540.976515:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161d79 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef585 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976617:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976632:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976644:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572540.976659:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc2:0x0]: returning size 3 00010000:00010000:0.0:1788572540.976761:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976769:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976777:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976783:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572540.976788:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976796:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976809:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:0.0:1788572540.976813:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572540.976818:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976824:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161d80 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976865:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161d79 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef585 expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976875:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572540.976878:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572540.976882:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161d79 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef585 expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572540.976891:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031600, rc 0) 00000100:00100000:0.0:1788572540.976963:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35897:x1875453360305920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 515us (6423us total) trans 4294970267 rc 301/301 00000100:00100000:1.0:1788572540.977167:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572540.977213:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572540.977230:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360306560 00000100:00100000:1.0:1788572540.977239:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360306432 00000100:00100000:1.0:1788572540.977247:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360306432 00000100:00100000:3.0:1788572540.977299:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360306560 00000100:00100000:0.0:1788572540.979819:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3940 00000100:00100000:1.0:1788572540.984456:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3941 00000100:00100000:1.0:1788572540.984491:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35899:x1875453360306432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572540.984520:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1788572540.984539:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3942 00010000:00010000:1.0:1788572540.984546:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161d8e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef593 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572540.984555:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35898:x1875453360306560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572540.984589:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb9:0x0] from 18446744073709551615 #0 00010000:00010000:1.0:1788572540.984661:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984678:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984691:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572540.984705:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbd:0x0]: returning size 3 00000100:00100000:0.0:1788572540.984777:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35898:x1875453360306560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 226us (7610us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572540.984786:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3942 00010000:00010000:1.0:1788572540.984830:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984840:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984857:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984865:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572540.984870:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984878:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984892:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627c7d65100) 00010000:00010000:1.0:1788572540.984894:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572540.984899:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984905:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161d95 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984958:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161d8e lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef593 expref: 84 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984968:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572540.984970:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572540.984975:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161d8e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef593 expref: 84 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572540.984984:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a91c00, rc 0) 00000100:00100000:1.0:1788572540.993741:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35899:x1875453360306432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9248us (16515us total) trans 4294970268 rc 301/301 00000100:00100000:1.0:1788572540.993755:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3941 00000100:00100000:0.0:1788572540.995332:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572540.995376:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572540.995445:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360306688 00000100:00100000:2.0:1788572540.995455:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360306688 00000100:00100000:2.0:1788572540.995519:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360306816 00000100:00100000:2.0:1788572540.995523:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360306816 00000100:00100000:2.0:1788572540.995555:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3943 00000100:00100000:2.0:1788572540.995565:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35897:x1875453360306688:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572540.995603:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xc2:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572540.995710:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35897:x1875453360306688:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 146us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.995718:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3943 00000100:00100000:2.0:1788572540.995736:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3944 00000100:00100000:2.0:1788572540.995743:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35898:x1875453360306816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572540.995753:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572540.995774:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161da3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef59a expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.995836:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.995850:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.995866:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.995880:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.995887:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.995893:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996224:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161db8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996234:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161db8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996241:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161db8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572540.996262:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:2.0:1788572540.996266:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572540.996284:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161db8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996296:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996302:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996313:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996319:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.996324:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996331:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996344:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572540.996347:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.996353:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996360:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161daa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996372:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996378:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996384:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996389:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572540.996392:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996401:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996409:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627e7a95b00) 00010000:00010000:2.0:1788572540.996411:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572540.996415:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996420:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161db1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996450:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161da3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef59a expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996472:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161db8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef59a expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572540.996480:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572540.996482:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572540.996484:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:2.0:1788572540.996517:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35898:x1875453360306816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 774us (1140us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572540.996524:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3944 00000100:00100000:0.0:1788572541.000772:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.000872:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360306944 00000100:00100000:2.0:1788572541.000885:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360306944 00000100:00100000:2.0:1788572541.001056:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3945 00000100:00100000:2.0:1788572541.001068:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35899:x1875453360306944:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572541.001111:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xbd:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572541.001231:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35899:x1875453360306944:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 165us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.001239:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3945 00000100:00080000:3.0:1788572541.002491:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572541.007203:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.007453:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.007825:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360307200 00000100:00100000:2.0:1788572541.007837:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360307200 00000100:00100000:2.0:1788572541.007903:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360307328 00000100:00100000:2.0:1788572541.007907:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360307328 00000100:00100000:2.0:1788572541.008157:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3946 00000100:00100000:2.0:1788572541.008170:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35898:x1875453360307200:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.008193:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.008196:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572541.008225:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161db8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef59a expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.008245:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572541.008250:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572541.008265:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161dbf lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5a1 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572541.008806:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x45:0x0], index 0: 0 00010000:00010000:2.0:1788572541.009074:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161dc6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.009089:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161dc6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.009099:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161dc6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.009161:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161dbf lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5a1 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.009176:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161dc6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5a1 expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.009184:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572541.009187:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572541.009189:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:2.0:1788572541.009258:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35898:x1875453360307200:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1091us (2065us total) trans 4294970269 rc 0/0 00000100:00100000:0.0:1788572541.009367:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3947 00000100:00100000:0.0:1788572541.009379:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35897:x1875453360307328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.009390:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.009402:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161dd4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5a8 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009449:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009473:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009481:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009495:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009502:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009508:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009737:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161de9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009746:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161de9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009753:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161de9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.009769:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:0.0:1788572541.009773:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.009784:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161de9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009793:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009804:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009810:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009816:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.009820:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009827:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.009836:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572541.009839:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.009843:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010485:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ddb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010501:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010507:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010512:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010517:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.010520:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010526:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010534:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627e7451600) 00010000:00010000:0.0:1788572541.010537:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.010540:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010547:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161de2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010589:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161dd4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5a8 expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010602:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161de9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5a8 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.010608:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572541.010610:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572541.010613:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246d800, rc 0) 00000100:00100000:0.0:1788572541.010661:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35897:x1875453360307328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1285us (3207us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.010671:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3947 00000100:00100000:2.0:1788572541.011761:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3946 00000100:00100000:0.0:1788572541.021858:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.021948:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360307584 00000100:00100000:2.0:1788572541.021960:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360307584 00000100:00100000:2.0:1788572541.022035:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3948 00000100:00100000:2.0:1788572541.022043:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35898:x1875453360307584:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.022096:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572541.022127:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161dc6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5a1 expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022143:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572541.022155:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161de9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5a8 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022162:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572541.022164:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572541.022183:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022192:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022199:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022207:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022212:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022216:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022408:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022414:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022419:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022422:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.022426:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022431:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022438:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572541.022440:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.022443:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022446:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161df0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022451:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022454:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022457:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022474:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.022477:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022483:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022491:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x6660d2c7 (ffff9627e7a95400) 00010000:00010000:2.0:1788572541.022493:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.022500:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.022506:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161df7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x6660d2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.023285:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35898:x1875453360307584:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1235us (1431us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572541.023295:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3948 00000100:00100000:3.0:1788572541.027696:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.027746:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.027840:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360307840 00000100:00100000:2.0:1788572541.027850:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360307840 00000100:00100000:2.0:1788572541.027899:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360307968 00000100:00100000:2.0:1788572541.027902:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360307968 00000100:00100000:2.0:1788572541.027926:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3949 00000100:00100000:2.0:1788572541.027936:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35899:x1875453360307840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.027966:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788572541.027985:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3950 00010000:00010000:2.0:1788572541.027987:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161dfe lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5af expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.027993:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35898:x1875453360307968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.028001:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.028034:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e05 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5b6 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.028061:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.028073:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.028084:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.028167:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e05 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.028187:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.028195:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:3.0:1788572541.028198:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572541.028201:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee600, rc 0) 00000100:00100000:3.0:1788572541.028252:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35898:x1875453360307968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 261us (505us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.028258:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3950 00000004:00080000:2.0:1788572541.028412:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x45:0x0], index 1: 0 00010000:00010000:2.0:1788572541.028623:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161e13 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.028632:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161e13 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.028640:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161e13 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.028694:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161dfe lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5af expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.028708:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161e13 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5af expref: 87 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.028715:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627e7a95600) 00010000:00010000:2.0:1788572541.028717:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572541.028720:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:2.0:1788572541.028758:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35899:x1875453360307840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 825us (1072us total) trans 4294970270 rc 0/0 00000100:00100000:2.0:1788572541.028765:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3949 00000100:00100000:0.0:1788572541.033912:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.033945:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.034211:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360308224 00000100:00100000:1.0:1788572541.034222:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360308224 00000100:00100000:1.0:1788572541.034284:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360308352 00000100:00100000:1.0:1788572541.034288:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360308352 00000100:00100000:1.0:1788572541.034371:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3951 00000100:00100000:1.0:1788572541.034381:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35898:x1875453360308224:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.034500:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.034520:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.034532:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572541.035124:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].57, 88 off12728 00000004:00010000:1.0:1788572541.035833:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000033fb0cdf reply state 00000000bcb062c0 transno 4294970271 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.035853:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572541.035959:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35898:x1875453360308224:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1578us (2059us total) trans 4294970271 rc 0/0 00000100:00100000:1.0:1788572541.035967:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3951 00000100:00100000:1.0:1788572541.035995:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3952 00000100:00100000:1.0:1788572541.036023:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35899:x1875453360308352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.036038:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.036053:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161e28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5bd expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036105:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036118:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036127:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036142:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036148:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036154:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036481:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161e3d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036492:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161e3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036501:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161e3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572541.036522:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 16384 00000004:00200000:1.0:1788572541.036525:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572541.036536:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161e3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036548:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036559:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036566:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036573:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.036577:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036585:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036596:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572541.036599:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.036604:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036611:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb161e2f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036623:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036629:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036635:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036640:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a90a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.036644:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036649:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036770:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd1d74aad (ffff9627d0ee2100) 00010000:00010000:1.0:1788572541.036773:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.036777:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036783:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161e36 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd1d74aad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036813:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161e28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5bd expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036824:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161e3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5bd expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.036832:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572541.036835:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572541.036837:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031600, rc 0) 00000100:00100000:1.0:1788572541.036865:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35899:x1875453360308352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 864us (2920us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.036874:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3952 00000100:00100000:3.0:1788572541.037032:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.037511:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360308608 00000100:00100000:3.0:1788572541.037520:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360308608 00000100:00100000:3.0:1788572541.037565:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3953 00000100:00100000:3.0:1788572541.037573:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35897:x1875453360308608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.037591:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.037603:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e44 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5c4 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572541.038872:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x46:0x0], index 0: 0 00010000:00010000:3.0:1788572541.041131:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb161e4b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.041157:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb161e4b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.041171:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb161e4b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.041249:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e44 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5c4 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.041570:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb161e4b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5c4 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.041582:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:3.0:1788572541.041585:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.041588:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:3.0:1788572541.041689:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35897:x1875453360308608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4115us (4656us total) trans 4294970272 rc 0/0 00000100:00100000:3.0:1788572541.041700:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3953 00000100:00100000:3.0:1788572541.043519:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.045229:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360308736 00000100:00100000:0.0:1788572541.045246:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360308736 00000100:00100000:0.0:1788572541.045527:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3954 00000100:00100000:0.0:1788572541.045540:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35898:x1875453360308736:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.045570:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.045593:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5cb expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.045665:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.045682:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.045702:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572541.045705:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572541.045709:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246d800, rc 0) 00000100:00100000:0.0:1788572541.045781:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35898:x1875453360308736:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 243us (2271us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.045789:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3954 00010000:00010000:1.0:1788572541.046103:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.046112:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.046120:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.046126:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.046131:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.046140:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.046152:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572541.046155:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572541.046158:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.046164:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161e21 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.049778:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.049874:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.050160:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360308992 00000100:00100000:1.0:1788572541.050167:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360309120 00000100:00100000:1.0:1788572541.050171:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360309120 00000100:00100000:2.0:1788572541.050171:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360308992 00000100:00100000:1.0:1788572541.050233:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1019 00000100:00100000:1.0:1788572541.050243:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360309120:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.050652:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360309120:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 410us (776us total) trans 4294967561 rc 0/0 00000100:00100000:1.0:1788572541.050734:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1019 00000100:00100000:2.0:1788572541.055532:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3955 00000100:00100000:2.0:1788572541.055563:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35899:x1875453360308992:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.055636:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572541.055662:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161e13 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5af expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055682:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627e7a95600) 00010000:00010000:2.0:1788572541.055697:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb161e3d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5bd expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055707:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572541.055710:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572541.055732:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055744:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055754:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055771:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055778:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.055784:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.057918:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572541.058635:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058650:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058657:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058664:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.058670:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058679:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058695:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572541.058698:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.058702:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058709:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161e60 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058722:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058727:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058733:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058738:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.058741:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058747:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058755:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x6709312a (ffff9627e7a95400) 00010000:00010000:2.0:1788572541.058757:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.058760:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.058765:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161e67 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x6709312a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.058906:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35899:x1875453360308992:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 3354us (9134us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572541.058916:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3955 00000100:00100000:2.0:1788572541.061425:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.061547:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360309376 00000100:00100000:1.0:1788572541.061559:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360309376 00000100:00100000:1.0:1788572541.061643:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 866 00000100:00100000:1.0:1788572541.061654:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35898:x1875453360309376:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.062225:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35898:x1875453360309376:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 571us (808us total) trans 4294970273 rc 0/0 00000100:00100000:1.0:1788572541.062233:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 866 00000100:00100000:3.0:1788572541.063580:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.063615:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.063732:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360309760 00000100:00100000:1.0:1788572541.063741:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360309760 00000100:00100000:1.0:1788572541.063787:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360309504 00000100:00100000:1.0:1788572541.063791:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360309504 00000100:00100000:1.0:1788572541.063839:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3956 00000100:00100000:1.0:1788572541.063848:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35897:x1875453360309760:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.063903:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.063929:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb161e4b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5c4 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.064237:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3957 00000100:00100000:3.0:1788572541.064248:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35899:x1875453360309504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.064258:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.064279:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb161e6e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5d2 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.064312:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.064326:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.064340:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.064425:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb161e6e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 94 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.064442:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.064451:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:3.0:1788572541.064454:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572541.064458:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:3.0:1788572541.064526:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35899:x1875453360309504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 280us (910us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.064533:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3957 00010000:00010000:1.0:1788572541.064869:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:1.0:1788572541.064878:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572541.064894:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 3/0,1 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.064903:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.064911:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.064924:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.064947:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.064955:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065171:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065179:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065186:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065192:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.065196:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065204:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065214:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572541.065217:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.065221:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 2/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065228:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb161e7c lrc: 0/0,0 mode: --/CW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065238:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065245:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065251:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065256:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.065259:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065265:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065273:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xa511ce7 (ffff9627d0ee2100) 00010000:00010000:1.0:1788572541.065276:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.065279:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.065285:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb161e83 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0xa511ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.065353:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35897:x1875453360309760:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1505us (1780us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572541.065360:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3956 00000100:00100000:1.0:1788572541.066710:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.066795:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.066910:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310016 00000100:00100000:0.0:1788572541.066920:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310016 00000100:00100000:2.0:1788572541.067164:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310272 00000100:00100000:2.0:1788572541.067170:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310272 00000100:00100000:3.0:1788572541.067259:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3958 00000100:00100000:3.0:1788572541.067271:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35898:x1875453360310016:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.067403:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35898:x1875453360310016:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 133us (697us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.067411:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3958 00000100:00100000:3.0:1788572541.067428:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3959 00000100:00100000:3.0:1788572541.067435:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35899:x1875453360310272:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.067513:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.067535:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.067545:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572541.068049:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].55, 88 off12624 00000004:00010000:3.0:1788572541.068144:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000781f7caf reply state 00000000e38b787e transno 4294970274 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.068155:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572541.068219:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35899:x1875453360310272:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 783us (1426us total) trans 4294970274 rc 0/0 00000100:00100000:3.0:1788572541.068226:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3959 00000100:00100000:3.0:1788572541.073539:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572541.073670:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.073683:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.073691:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.073698:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.073703:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.073712:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.073729:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:2.0:1788572541.073732:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572541.073737:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.073743:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161e8a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.073806:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310400 00000100:00100000:1.0:1788572541.073815:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310400 00000100:00100000:1.0:1788572541.073870:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3960 00000100:00100000:1.0:1788572541.073881:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35898:x1875453360310400:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.073901:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.073921:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb161e91 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5d9 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.074041:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e98 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.074055:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e98 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.074066:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e98 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572541.074104:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572541.074124:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e98 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.074151:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb161e91 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d9 expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.074166:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e98 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d9 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.074173:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572541.074176:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572541.074178:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc 0) 00000100:00100000:1.0:1788572541.074236:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35898:x1875453360310400:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 357us (710us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.074247:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3960 00000100:00100000:1.0:1788572541.075316:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.075379:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310528 00000100:00100000:3.0:1788572541.075386:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310528 00000100:00100000:3.0:1788572541.075436:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3961 00000100:00100000:3.0:1788572541.075445:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35897:x1875453360310528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.075469:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.075489:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161e9f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5e0 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.075520:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.075532:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.075547:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.075646:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161e9f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.075661:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 94 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.075669:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:3.0:1788572541.075672:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572541.075675:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ae0a00, rc 0) 00000100:00100000:3.0:1788572541.075735:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35897:x1875453360310528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 291us (424us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.075741:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3961 00000100:00100000:3.0:1788572541.077624:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.078248:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310656 00000100:00100000:0.0:1788572541.078260:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310656 00000100:00100000:0.0:1788572541.078326:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3962 00000100:00100000:0.0:1788572541.078336:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35899:x1875453360310656:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.078363:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.078386:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5e7 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.078458:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.078484:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.078504:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:0.0:1788572541.078506:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572541.078511:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246c000, rc 0) 00000100:00100000:0.0:1788572541.078584:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35899:x1875453360310656:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 249us (967us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.078591:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3962 00000100:00100000:0.0:1788572541.085590:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.085739:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310912 00000100:00100000:2.0:1788572541.085750:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310912 00000100:00100000:2.0:1788572541.085852:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1020 00000100:00100000:2.0:1788572541.085863:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360310912:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.086221:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360310912:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 348us (630us total) trans 4294967562 rc 0/0 00000100:00100000:2.0:1788572541.086233:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1020 00000100:00100000:1.0:1788572541.087490:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.088434:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360310784 00000100:00100000:0.0:1788572541.088444:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360310784 00000100:00100000:0.0:1788572541.088538:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1021 00000100:00100000:0.0:1788572541.088549:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360310784:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.088569:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.088591:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5ee expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.088619:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x45:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737c, blocks: 0x1 00010000:00010000:0.0:1788572541.088639:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=889s v=(0 0) 00010000:00010000:0.0:1788572541.088644:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.088655:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.088676:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.088690:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:0.0:1788572541.088743:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360310784:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 197us (1259us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.088751:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1021 00000100:00100000:2.0:1788572541.089452:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.090488:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360311168 00000100:00100000:1.0:1788572541.090499:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360311168 00000100:00100000:1.0:1788572541.090590:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 867 00000100:00100000:1.0:1788572541.090600:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35899:x1875453360311168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.091337:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35899:x1875453360311168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 736us (1886us total) trans 4294970275 rc 0/0 00000100:00100000:1.0:1788572541.091344:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 867 00000100:00100000:2.0:1788572541.091443:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.091566:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.091590:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360311424 00000100:00100000:2.0:1788572541.091596:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360311424 00000100:00100000:2.0:1788572541.091634:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360311680 00000100:00100000:2.0:1788572541.091637:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360311680 00000100:00100000:2.0:1788572541.091655:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3963 00000100:00100000:2.0:1788572541.091663:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35897:x1875453360311424:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.091736:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.091752:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.091762:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572541.092074:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].58, 88 off12816 00000004:00010000:2.0:1788572541.092129:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001a245f2d reply state 00000000e737bcc9 transno 4294970276 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.092136:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572541.092188:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35897:x1875453360311424:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 526us (744us total) trans 4294970276 rc 0/0 00000100:00100000:2.0:1788572541.092192:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3963 00000100:00100000:2.0:1788572541.092207:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3964 00000100:00100000:2.0:1788572541.092212:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec191c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35898:x1875453360311680:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.092245:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec191c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35898:x1875453360311680:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 34us (681us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.092249:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3964 00010000:00010000:1.0:1788572541.100300:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.100319:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.100326:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.100332:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.100338:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.100347:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.100365:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:1.0:1788572541.100368:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572541.100374:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.100381:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161ebb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.100663:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.100753:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.100983:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312064 00000100:00100000:1.0:1788572541.100992:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312064 00000100:00100000:1.0:1788572541.101075:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1022 00000100:00100000:1.0:1788572541.101086:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:35898:x1875453360312064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.101105:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.101124:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5f5 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.101156:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.101164:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.101168:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.101174:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.101208:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.101230:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.101252:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382344448:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.101339:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788a600 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.102407:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360311936 00000100:00100000:3.0:1788572541.102416:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360311936 00000100:00100000:3.0:1788572541.102483:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3965 00000100:00100000:3.0:1788572541.102492:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35899:x1875453360311936:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.102610:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35899:x1875453360311936:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 119us (1952us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.102618:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3965 00000100:00100000:2.0:1788572541.107188:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.107326:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312192 00000100:00100000:0.0:1788572541.107335:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312192 00000100:00100000:0.0:1788572541.107394:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3966 00000100:00100000:0.0:1788572541.107406:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35897:x1875453360312192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.107430:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.107449:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef5fc expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.107532:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 98 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.107547:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 98 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.107568:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:0.0:1788572541.107571:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572541.107574:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:0.0:1788572541.107640:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35897:x1875453360312192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 235us (462us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.107647:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3966 00000100:00100000:1.0:1788572541.107725:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382344448:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.107746:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.107757:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47e00, rc 0) 00000100:00100000:1.0:1788572541.107774:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35898:x1875453360312064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6692us (7022us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.107779:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1022 00000100:00100000:2.0:1788572541.108761:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.108797:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.108870:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312320 00000100:00100000:1.0:1788572541.108877:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312320 00010000:00010000:1.0:1788572541.108907:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.108950:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 940 00000100:00100000:1.0:1788572541.108960:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e50050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360312320:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.108973:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.108994:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.109038:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.109067:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb161eb4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5ee expref: 13 pid: 6682 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.109084:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.109088:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.109096:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.109106:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.109143:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.109159:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382344576:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572541.109340:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788a600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572541.111549:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312448 00000100:00100000:0.0:1788572541.111562:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312448 00000100:00100000:0.0:1788572541.111628:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3967 00000100:00100000:0.0:1788572541.111639:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35899:x1875453360312448:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.111658:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.111680:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161ed0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef603 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.111746:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161ed7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.111757:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161ed7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.111766:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161ed7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.111796:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.111812:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161ed7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.111838:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb161ed0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef603 expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.111860:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161ed7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef603 expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.111869:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:0.0:1788572541.111872:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572541.111875:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bace00, rc 0) 00000100:00100000:0.0:1788572541.111933:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35899:x1875453360312448:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 297us (3136us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.111939:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3967 00000100:00100000:0.0:1788572541.111990:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.112149:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312576 00000100:00100000:1.0:1788572541.112152:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312576 00000100:00100000:1.0:1788572541.112178:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1023 00000100:00100000:1.0:1788572541.112183:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360312576:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.112407:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360312576:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 223us (415us total) trans 4294967562 rc 0/0 00000100:00100000:1.0:1788572541.112412:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1023 00000100:00100000:0.0:1788572541.113572:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.113664:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382344576:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572541.113687:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.113731:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e50050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360312320:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 4773us (4975us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.113742:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 940 00000100:00100000:2.0:1788572541.114777:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312704 00000100:00100000:2.0:1788572541.114787:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312704 00002000:00100000:2.0:1788572541.114833:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x45:0x0] (0->2) req@ffff9626c1b59500 x1875453360312704/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.114843:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b59500 with x1875453360312704 ext(0->2) 00002000:00010000:2.0:1788572541.114902:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b59500 00000100:00100000:2.0:1788572541.115035:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 232 00000100:00100000:2.0:1788572541.115045:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360312704:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.115538:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360312704:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 494us (1969us total) trans 4294967563 rc 0/0 00000100:00100000:2.0:1788572541.115546:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 232 00002000:00100000:2.0:1788572541.115554:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x45:0x0] (0->2) req@ffff9626c1b59500 x1875453360312704/t4294967563(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/448 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.115561:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b59500 with x1875453360312704 ext(0->2) 00002000:00010000:2.0:1788572541.115564:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b59500 00000100:00100000:0.0:1788572541.116670:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.116739:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312832 00000100:00100000:3.0:1788572541.116747:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312832 00000100:00100000:3.0:1788572541.116831:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1024 00000100:00100000:3.0:1788572541.116841:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360312832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.116855:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.116876:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 2/0,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef60a expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572541.116904:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x45:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737c, blocks: 0x1 00010000:00010000:3.0:1788572541.116922:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=889s v=(0 0) 00010000:00010000:3.0:1788572541.116927:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572541.116948:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.116966:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.116979:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ae1600, rc 0) 00000100:00100000:3.0:1788572541.117057:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360312832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 218us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.117064:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1024 00000100:00100000:3.0:1788572541.121852:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.121906:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.122038:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360312960 00000100:00100000:0.0:1788572541.122051:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360312960 00000100:00100000:0.0:1788572541.122117:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3968 00000100:00100000:0.0:1788572541.122127:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35898:x1875453360312960:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.122268:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360313216 00000100:00100000:1.0:1788572541.122273:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360313216 00000100:00100000:1.0:1788572541.122302:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 868 00000100:00100000:1.0:1788572541.122310:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35897:x1875453360313216:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.122749:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35897:x1875453360313216:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 437us (840us total) trans 4294970277 rc 0/0 00000100:00100000:1.0:1788572541.122758:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 868 00000100:00100000:0.0:1788572541.136361:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.136473:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360313472 00000100:00100000:1.0:1788572541.136482:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360313472 00000100:00100000:1.0:1788572541.136543:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3969 00000100:00100000:1.0:1788572541.136552:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35899:x1875453360313472:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.136661:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35899:x1875453360313472:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 110us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.136669:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3969 00000100:00100000:2.0:1788572541.141619:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.141747:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360313728 00000100:00100000:2.0:1788572541.141757:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360313728 00000100:00100000:2.0:1788572541.141809:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3970 00000100:00100000:2.0:1788572541.141818:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35897:x1875453360313728:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.141924:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35897:x1875453360313728:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 106us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.141933:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3970 00000100:00100000:0.0:1788572541.147906:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.149215:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360313856 00000100:00100000:1.0:1788572541.149231:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360313856 00000100:00100000:1.0:1788572541.149294:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1025 00000100:00100000:1.0:1788572541.149304:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35899:x1875453360313856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.149322:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.149343:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef611 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.149384:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.149394:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.149399:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.149405:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.149442:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.149457:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.149486:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382344704:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.149568:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788acc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.167568:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.167665:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382344704:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.167698:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.167720:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:1.0:1788572541.167782:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35899:x1875453360313856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 18479us (19885us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.167795:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1025 00000100:00100000:3.0:1788572541.170392:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360313984 00000100:00100000:3.0:1788572541.170407:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360313984 00000100:00100000:3.0:1788572541.170489:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3971 00000100:00100000:3.0:1788572541.170500:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35897:x1875453360313984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.170525:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.170547:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161eec lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef618 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.170622:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161ef3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.170637:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161ef3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.170649:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161ef3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.170688:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.170708:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161ef3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.170737:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161eec lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef618 expref: 103 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.170752:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161ef3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef618 expref: 102 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.170760:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:3.0:1788572541.170763:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572541.170766:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ae1200, rc 0) 00000100:00100000:3.0:1788572541.170818:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35897:x1875453360313984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 321us (3261us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.170826:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3971 00000100:00100000:2.0:1788572541.187612:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.187720:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.187750:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360314240 00000100:00100000:2.0:1788572541.187759:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360314240 00000100:00100000:2.0:1788572541.187808:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1026 00000100:00100000:2.0:1788572541.187817:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360314240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.187835:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.187853:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef61f expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572541.187880:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x46:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737c, blocks: 0x1 00010000:00010000:2.0:1788572541.187899:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=889s v=(0 0) 00010000:00010000:2.0:1788572541.187902:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.187911:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.187927:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.187945:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:2.0:1788572541.187997:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360314240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 178us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.188032:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1026 00000100:00100000:1.0:1788572541.188357:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360314112 00000100:00100000:1.0:1788572541.188363:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360314112 00010000:00010000:1.0:1788572541.188387:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.188422:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 941 00000100:00100000:1.0:1788572541.188430:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360314112:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.188442:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.188470:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.188483:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.188501:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161ede lrc: 0/0,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef60a expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.188515:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.188520:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.188527:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.188536:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.188577:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.188590:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382344832:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572541.188653:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788acc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.194911:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382344832:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572541.194940:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.195018:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360314112:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6574us (7402us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.195031:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 941 00000100:00100000:1.0:1788572541.199978:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.201561:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.203228:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360314624 00000100:00100000:1.0:1788572541.203242:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360314624 00000100:00100000:1.0:1788572541.203298:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3972 00000100:00100000:1.0:1788572541.203307:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35897:x1875453360314624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.203411:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35897:x1875453360314624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 104us (1858us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.203418:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3972 00000100:00100000:2.0:1788572541.205489:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360314368 00000100:00100000:2.0:1788572541.205503:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360314368 00002000:00100000:2.0:1788572541.205552:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x45:0x0] (0->2) req@ffff9626c13bce00 x1875453360314368/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.205565:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bce00 with x1875453360314368 ext(0->2) 00002000:00010000:2.0:1788572541.205571:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bce00 00000100:00100000:2.0:1788572541.205670:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 233 00000100:00100000:2.0:1788572541.205683:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360314368:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.206276:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360314368:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 595us (6306us total) trans 4294967563 rc 0/0 00000100:00100000:2.0:1788572541.206285:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 233 00002000:00100000:2.0:1788572541.206293:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x45:0x0] (0->2) req@ffff9626c13bce00 x1875453360314368/t4294967563(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/448 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.206302:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bce00 with x1875453360314368 ext(0->2) 00002000:00010000:2.0:1788572541.206305:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bce00 00000100:00100000:1.0:1788572541.213855:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.213888:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360314752 00000100:00100000:1.0:1788572541.213897:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360314752 00000100:00100000:1.0:1788572541.213949:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1027 00000100:00100000:1.0:1788572541.213958:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35897:x1875453360314752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.213976:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.213994:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef626 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.214048:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.214058:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.214062:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.214069:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 17 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.214103:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.214116:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 17 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.214129:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382344960:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.215411:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788acc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.215550:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.215646:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360314880 00000100:00100000:1.0:1788572541.216310:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360314880 00000100:00100000:0.0:1788572541.216478:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3973 00000100:00100000:0.0:1788572541.216491:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35899:x1875453360314880:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572541.216619:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35899:x1875453360314880:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 130us (1071us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.216626:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3973 00000100:00100000:3.0:1788572541.222480:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.222512:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.222592:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315008 00000100:00100000:1.0:1788572541.222602:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315008 00010000:00010000:1.0:1788572541.222699:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.222750:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 942 00000100:00100000:1.0:1788572541.222760:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360315008:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.222796:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.222860:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.222873:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.222897:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.222901:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.222909:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 18 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.222918:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 18 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.222955:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 18 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.222969:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382345088:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572541.223278:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3251480 going to sleep for 11 seconds 00010000:00010000:1.0:1788572541.223319:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb161efa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef61f expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.223333:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382344960:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.223357:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 17 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.223368:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:1.0:1788572541.223511:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35897:x1875453360314752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9552us (9664us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.223519:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1027 00000100:00100000:1.0:1788572541.223862:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315136 00000100:00100000:1.0:1788572541.223868:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315136 00000100:00100000:1.0:1788572541.223906:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1028 00000100:00100000:1.0:1788572541.223913:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360315136:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572541.226846:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35898:x1875453360312960:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 104709us (104994us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.226861:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3968 00000100:00100000:1.0:1788572541.228149:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382345088:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572541.228179:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.228216:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360315008:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5457us (5757us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.228227:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 942 00000100:00100000:3.0:1788572541.229087:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000001:00080000:2.0:1788572541.229104:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970267 is committed 00000020:00080000:2.0:1788572541.229126:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ca87f840 refcount 2 committed 0 result 0 batchid 102 00000020:00080000:2.0:1788572541.229131:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cea500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ca87f000 00000020:00080000:2.0:1788572541.229135:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceae00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff092c00 00000100:00100000:1.0:1788572541.229181:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315264 00000100:00100000:1.0:1788572541.229190:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315264 00000020:00080000:3.0:1788572541.229202:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 96 00000100:00100000:1.0:1788572541.229578:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1029 00000100:00100000:1.0:1788572541.229591:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360315264:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000020:00080000:2.0:1788572541.229801:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627e75b8180 refcount 2 committed 0 result 0 batchid 101 00000020:00080000:2.0:1788572541.229806:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cea400 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627e75b8540 00000020:00080000:2.0:1788572541.229809:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceac00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627fb4de400 00000020:00080000:1.0:1788572541.229843:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 96 00000020:00080000:2.0:1788572541.229879:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 99 batchid updated 00000020:00080000:1.0:1788572541.229907:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 99 00000020:00080000:1.0:1788572541.229922:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 99 00000001:00080000:2.0:1788572541.230048:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970277 is committed 00000020:00080000:2.0:1788572541.230091:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ff3fe0c0 refcount 2 committed 0 result 0 batchid 100 00000020:00080000:2.0:1788572541.230095:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627fb363f00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff3fee40 00000020:00080000:2.0:1788572541.230098:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627fb363000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627caa5e800 00000020:00080000:2.0:1788572541.230134:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 102 committed batchid 99 00000020:00080000:2.0:1788572541.230319:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 99 00000100:00100000:0.0:1788572541.236580:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.237335:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315392 00000100:00100000:1.0:1788572541.237350:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315392 00002000:00100000:1.0:1788572541.237406:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x46:0x0] (0->2) req@ffff9626c1b58a80 x1875453360315392/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572541.237418:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b58a80 with x1875453360315392 ext(0->2) 00002000:00010000:1.0:1788572541.237430:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b58a80 00000100:00100000:1.0:1788572541.237479:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 234 00000100:00100000:1.0:1788572541.237491:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2349:x1875453360315392:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572541.238249:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360315392:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 757us (1675us total) trans 4294967564 rc 0/0 00000100:00100000:0.0:1788572541.238262:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 234 00002000:00100000:0.0:1788572541.238342:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x46:0x0] (0->2) req@ffff9626c1b58a80 x1875453360315392/t4294967564(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/448 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572541.238351:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b58a80 with x1875453360315392 ext(0->2) 00002000:00010000:0.0:1788572541.238356:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b58a80 00000100:00100000:3.0:1788572541.244431:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.244998:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315520 00000100:00100000:1.0:1788572541.245039:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315520 00000100:00100000:1.0:1788572541.245103:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3974 00000100:00100000:1.0:1788572541.245115:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35897:x1875453360315520:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.245240:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35897:x1875453360315520:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 128us (820us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.245248:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3974 00000100:00100000:3.0:1788572541.252056:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.252202:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315648 00000100:00100000:1.0:1788572541.252216:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315648 00000100:00100000:1.0:1788572541.252325:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1030 00000100:00100000:1.0:1788572541.252337:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360315648:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:3.0:1788572541.269323:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967561 is committed 00000001:00080000:3.0:1788572541.269342:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967559 is committed 00000001:00080000:3.0:1788572541.269351:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967563 is committed 00000100:00100000:1.0:1788572541.269426:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360315136:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 45506us (46906us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.269437:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1028 00000100:00100000:0.0:1788572541.276611:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.276655:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360315904 00000100:00100000:0.0:1788572541.276667:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360315904 00000100:00100000:1.0:1788572541.280628:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3975 00000100:00100000:1.0:1788572541.280658:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35899:x1875453360315904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.280732:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.280762:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161ed7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef603 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.280784:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:1.0:1788572541.280788:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572541.280818:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.280831:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.280840:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572541.281112:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003a370dac reply state 00000000c9419a20 transno 4294970278 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.281122:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572541.281236:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35899:x1875453360315904:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 593us (4636us total) trans 4294970278 rc 0/0 00000100:00100000:1.0:1788572541.281244:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3975 00010000:00010000:1.0:1788572541.285502:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.285522:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.285531:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.285543:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.285548:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.285557:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.285576:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:1.0:1788572541.285579:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.285584:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.285591:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb161f08 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.286657:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.286737:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360316032 00000100:00100000:1.0:1788572541.286749:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360316032 00000100:00100000:1.0:1788572541.286815:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1031 00000100:00100000:1.0:1788572541.286826:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360316032:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572541.287133:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x45:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572541.287137:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x45:0x0] updating lvb mtime from disk: 0 -> 1788572515 00002000:00010000:1.0:1788572541.287140:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x45:0x0] updating lvb atime from disk: 0 -> 1788572515 00002000:00010000:1.0:1788572541.287142:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x45:0x0] updating lvb ctime from disk: 1788572540 -> 1788572527 00002000:00010000:1.0:1788572541.287145:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x45:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572541.287209:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360316032:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 385us (559us total) trans 4294967564 rc 0/0 00000100:00100000:1.0:1788572541.287219:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1031 00000100:00100000:0.0:1788572541.302660:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.302735:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360316288 00000100:00100000:0.0:1788572541.302746:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360316288 00000100:00100000:0.0:1788572541.302804:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3976 00000100:00100000:0.0:1788572541.302815:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35899:x1875453360316288:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.302893:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.302912:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.302921:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.302960:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.302975:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 65 pid: 5839 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.302992:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382345216:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572541.303101:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.306123:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382345216:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.306155:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.308104:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.308963:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360316416 00000100:00100000:2.0:1788572541.308977:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360316416 00010000:00010000:2.0:1788572541.309052:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 66 pid: 5839 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.309093:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 943 00000100:00100000:2.0:1788572541.309105:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360316416:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.309120:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572541.309128:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 66 pid: 5839 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.309141:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 66 pid: 5839 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.309170:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb161e75 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d2 expref: 66 pid: 5839 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.309185:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:2.0:1788572541.309191:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.309201:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.309209:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.309620:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572541.309702:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360316416:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 599us (1608us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.309711:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 943 00000001:00080000:2.0:1788572541.310817:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967562 is committed 00000001:00080000:2.0:1788572541.310834:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967557 is committed 00000001:00080000:2.0:1788572541.310854:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967563 is committed 00000001:00080000:2.0:1788572541.310875:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967560 is committed 00000100:00100000:1.0:1788572541.313103:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360315264:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 83503us (84009us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.313118:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1029 00010000:00010000:3.0:1788572541.314088:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314105:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314140:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314148:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314156:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314162:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.314168:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314175:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314193:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:3.0:1788572541.314197:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572541.314204:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.314221:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f0f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.314306:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35899:x1875453360316288:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 11489us (11665us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572541.314317:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3976 00000100:00100000:0.0:1788572541.327302:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.327594:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360316544 00000100:00100000:3.0:1788572541.327605:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360316544 00000100:00100000:3.0:1788572541.327667:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1032 00000100:00100000:3.0:1788572541.327678:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360316544:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.327727:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.327749:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.327759:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572541.327763:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572541.327768:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.327807:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.327821:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 11 pid: 10416 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.327836:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382345344:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572541.327917:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.331144:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.331294:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360316800 00000100:00100000:0.0:1788572541.331305:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360316800 00000100:00100000:0.0:1788572541.331364:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3977 00000100:00100000:0.0:1788572541.331375:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35898:x1875453360316800:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.331432:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572541.331470:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb161e98 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5d9 expref: 64 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.331491:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572541.331495:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572541.331529:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.331544:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.331554:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572541.331706:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f6de583d reply state 00000000ba218eaa transno 4294970279 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.331716:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572541.331814:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35898:x1875453360316800:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 440us (679us total) trans 4294970279 rc 0/0 00000100:00100000:0.0:1788572541.331823:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3977 00000100:00100000:1.0:1788572541.334939:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.334990:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.335037:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360316928 00000100:00100000:1.0:1788572541.335046:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360316928 00010000:00010000:1.0:1788572541.335086:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 12 pid: 10416 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.335122:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 944 00000100:00100000:1.0:1788572541.335132:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360316928:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.335146:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.335169:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 12 pid: 10416 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335183:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 12 pid: 10416 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335208:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572541.335212:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572541.335218:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335235:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335256:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.335306:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360316928:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 176us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.335314:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 944 00010000:00010000:1.0:1788572541.335341:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335351:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335357:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335363:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.335368:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335376:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335388:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572541.335391:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.335396:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.335402:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb161f1d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.335969:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb161ee5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef611 expref: 11 pid: 10416 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.335985:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382345344:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572541.336029:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336046:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336054:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336079:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336086:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336094:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336100:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572541.336104:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336112:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336124:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.336136:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fe46a428/0x85ace67feb161f16 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.336196:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360316544:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 8520us (8905us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.336205:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1032 00000100:00100000:2.0:1788572541.336224:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360317056 00000100:00100000:2.0:1788572541.336228:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360317056 00000100:00100000:2.0:1788572541.336256:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1033 00000100:00100000:2.0:1788572541.336263:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360317056:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572541.336500:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572541.336504:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:2.0:1788572541.336506:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:2.0:1788572541.336508:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb ctime from disk: 1788572540 -> 1788572527 00002000:00010000:2.0:1788572541.336510:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572541.336534:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360317056:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 270us (1543us total) trans 4294967565 rc 0/0 00000100:00100000:2.0:1788572541.336541:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1033 00000100:00100000:1.0:1788572541.345606:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360315648:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 93260us (93551us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.345622:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1030 00000001:00080000:3.0:1788572541.345719:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967564 is committed 00000100:00100000:1.0:1788572541.347655:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.347767:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360317312 00000100:00100000:3.0:1788572541.347781:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360317312 00000100:00100000:3.0:1788572541.347856:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3978 00000100:00100000:3.0:1788572541.347867:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35898:x1875453360317312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.347963:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.347983:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.347994:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.348069:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.348085:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 64 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.348100:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382345472:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572541.348192:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3d4a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572541.351484:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.351530:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.351676:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360317824 00000100:00100000:2.0:1788572541.351687:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360317824 00000100:00100000:2.0:1788572541.351743:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360317568 00000100:00100000:2.0:1788572541.351747:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360317568 00000100:00100000:2.0:1788572541.351775:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3979 00000100:00100000:2.0:1788572541.351786:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35897:x1875453360317824:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.351850:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572541.351876:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161ef3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef618 expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.351897:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:2.0:1788572541.351903:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572541.351949:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.351964:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.351975:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572541.352189:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ba3c82c6 reply state 000000006cd30376 transno 4294970280 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.352199:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572541.352292:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35897:x1875453360317824:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 508us (818us total) trans 4294970280 rc 0/0 00000100:00100000:2.0:1788572541.352301:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3979 00000100:00100000:2.0:1788572541.352322:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3980 00000100:00100000:2.0:1788572541.352331:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35899:x1875453360317568:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.352374:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352384:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352396:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352559:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352570:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352579:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352614:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.352627:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 67 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.352642:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382345600:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572541.352702:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.355088:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382345472:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.355120:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.359194:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.359390:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360317952 00000100:00100000:1.0:1788572541.359401:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360317952 00010000:00010000:1.0:1788572541.359445:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 68 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.359502:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 945 00000100:00100000:1.0:1788572541.359512:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360317952:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.359527:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.359534:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 68 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.359547:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 68 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.359569:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb161e0c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5b6 expref: 68 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.359584:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572541.359590:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.359599:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.359606:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.359652:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.359711:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360317952:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 200us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.359718:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 945 00010000:00010000:3.0:1788572541.359800:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359809:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359839:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359847:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359854:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359859:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.359864:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359871:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359880:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:3.0:1788572541.359883:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572541.359887:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.359893:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161f24 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.359933:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35898:x1875453360317312:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 12068us (12302us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572541.359943:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3978 00010000:00010000:2.0:1788572541.360601:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.360611:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.360616:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.360626:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.360630:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.360636:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.360728:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:2.0:1788572541.360731:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.360735:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.360741:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f2b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.362435:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.362913:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382345600:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572541.362941:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.363138:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360318080 00000100:00100000:1.0:1788572541.363147:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360318080 00000100:00100000:1.0:1788572541.363285:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1034 00000100:00100000:1.0:1788572541.363296:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360318080:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572541.363527:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572541.363531:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb mtime from disk: 0 -> 1788572515 00002000:00010000:1.0:1788572541.363534:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb atime from disk: 0 -> 1788572515 00002000:00010000:1.0:1788572541.363536:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb ctime from disk: 1788572540 -> 1788572527 00002000:00010000:1.0:1788572541.363539:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572541.363595:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360318080:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 300us (1167us total) trans 4294967566 rc 0/0 00000100:00100000:1.0:1788572541.363603:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1034 00000100:00100000:1.0:1788572541.366986:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.369661:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.369805:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360318336 00000100:00100000:1.0:1788572541.369817:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360318336 00010000:00010000:1.0:1788572541.369908:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 65 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.369960:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 946 00000100:00100000:1.0:1788572541.369969:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360318336:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.369985:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.369992:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 65 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.370044:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 65 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.370070:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb161ead lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e7 expref: 65 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.370088:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:1.0:1788572541.370093:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.370102:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.370109:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.370134:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.370196:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360318336:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 228us (543us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.370204:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 946 00010000:00010000:0.0:1788572541.370724:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.370734:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572541.370946:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970281 00000100:00100000:0.0:1788572541.373346:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.373423:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360318592 00000100:00100000:1.0:1788572541.373436:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360318592 00000100:00100000:1.0:1788572541.373517:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3981 00000100:00100000:1.0:1788572541.373528:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35897:x1875453360318592:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.373605:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.373623:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.373633:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.373684:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.373699:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 65 pid: 5837 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.373715:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382345728:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.373797:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572541.375621:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360318208 00000100:00100000:2.0:1788572541.375632:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360318208 00000100:00100000:1.0:1788572541.376195:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382345728:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.376222:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.376621:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.376680:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360318720 00000100:00100000:1.0:1788572541.376686:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360318720 00010000:00010000:1.0:1788572541.376759:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 66 pid: 5837 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.376807:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 947 00000100:00100000:1.0:1788572541.376818:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e53c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360318720:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.376835:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.376842:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 66 pid: 5837 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.376856:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 66 pid: 5837 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.376882:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161ea6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5e0 expref: 66 pid: 5837 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.376900:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:1.0:1788572541.376905:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.376914:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.376922:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.376946:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.376994:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e53c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360318720:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 179us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.377025:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 947 00010000:00010000:1.0:1788572541.377077:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377084:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377111:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377118:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377124:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377130:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.377134:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377142:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377151:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:1.0:1788572541.377154:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572541.377170:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.377176:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f40 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.377215:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35897:x1875453360318592:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 3689us (3878us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572541.377223:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3981 00000001:00080000:3.0:1788572541.378823:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970282 00000001:00080000:3.0:1788572541.378966:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970283 00000100:00100000:3.0:1788572541.379095:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12edc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382345856:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572541.379279:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572541.379321:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c11cac00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572541.379390:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382345856 00000100:00100000:0.0:1788572541.379397:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382345856 00000100:00100000:0.0:1788572541.379456:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1788572541.379475:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382345856:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572541.379562:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382345856:12345-0@lo:1000:mdt00_004.0 Request processed in 89us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.379570:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:1.0:1788572541.379646:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967561 gen 2 00000004:00080000:1.0:1788572541.379653:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967560 committed transno 4294967561 00000004:00080000:1.0:1788572541.379670:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967561 committed transno 4294967561 00000100:00100000:1.0:1788572541.379686:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12edc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382345856:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572541.379738:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.379839:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360318848 00000100:00100000:3.0:1788572541.379844:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360318848 00000100:00100000:3.0:1788572541.379874:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1035 00000100:00100000:3.0:1788572541.379882:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7660050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360318208:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.379910:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1036 00010000:00010000:3.0:1788572541.379918:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.379920:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360318848:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.379936:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.379938:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 14 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.379947:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.379947:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572541.379951:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572541.379956:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.379959:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.379961:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572541.379965:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 4/1,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.379983:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.379983:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 14 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.379996:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 14 pid: 13272 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.380297:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382345984:192.168.202.40@tcp:104: 00000001:00080000:1.0:1788572541.381175:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970284 00000100:00100000:3.0:1788572541.381325:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c1370780 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.381684:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13be300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382346240:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572541.381898:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572541.382124:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff962800801780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.382286:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382346240 00000100:00100000:3.0:1788572541.382296:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382346240 00000100:00100000:3.0:1788572541.382369:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1788572541.382381:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382346240:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572541.382757:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382346240:12345-0@lo:1000:mdt00_004.0 Request processed in 258us (738us total) trans 4294967565 rc 0/0 00000100:00100000:3.0:1788572541.382768:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:1.0:1788572541.382942:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13be300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382346240:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572541.382984:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.382996:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383040:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383048:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.383052:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383061:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383078:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d599a700) 00010000:00010000:1.0:1788572541.383081:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572541.383085:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383095:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161f39 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x7c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383127:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383133:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383139:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383144:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d4c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.383147:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383154:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383404:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627c7d65100) 00010000:00010000:1.0:1788572541.383407:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572541.383411:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383417:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4c00/0x85ace67feb15f07e lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383441:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383454:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383475:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383483:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383489:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383494:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383499:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.383503:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383509:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383515:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627c7d65100) 00010000:00010000:1.0:1788572541.383517:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.383520:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383526:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f55 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383537:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383542:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383548:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383553:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.383555:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383561:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383567:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627c7d65100) 00010000:00010000:1.0:1788572541.383569:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572541.383572:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.383577:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161f32 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.384588:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35899:x1875453360317568:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 32249us (33047us total) trans 4294970285 rc 0/0 00000100:00100000:1.0:1788572541.384601:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3980 00000020:00080000:1.0:1788572541.384607:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00010000:00010000:2.0:1788572541.386588:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 14 pid: 6679 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.386618:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382346112:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572541.386889:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572541.394408:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.394484:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.394556:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.394631:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360319232 00000100:00100000:2.0:1788572541.394639:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360319232 00000100:00100000:3.0:1788572541.394663:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382346112:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572541.394692:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.394727:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382345984:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572541.394740:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394745:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 15 pid: 13272 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.394774:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360319360 00000100:00100000:2.0:1788572541.394777:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360319360 00000100:00100000:3.0:1788572541.394782:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 948 00010000:00010000:2.0:1788572541.394790:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 16 pid: 6679 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.394792:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360319232:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572541.394809:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1788572541.394819:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 949 00000100:00100000:2.0:1788572541.394825:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360319360:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.394832:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572541.394833:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 16 pid: 13272 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394842:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 16 pid: 6679 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.394846:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 16 pid: 13272 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394852:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 16 pid: 6679 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394877:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb161f01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef626 expref: 16 pid: 6679 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.394877:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb161ec2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x45:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5f5 expref: 16 pid: 13272 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394895:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572541.394895:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572541.394900:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572541.394900:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572541.394908:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.394908:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394920:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.394920:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.394944:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572541.394944:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572541.394990:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360319360:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 165us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.394998:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 949 00010000:00010000:2.0:1788572541.395049:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.395056:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360319104 00010000:00010000:2.0:1788572541.395058:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.395061:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360319104 00010000:00010000:2.0:1788572541.395086:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.395091:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 869 00010000:00010000:2.0:1788572541.395095:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.395098:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35899:x1875453360319104:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.395103:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.395109:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572541.395113:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.395121:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.395131:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.395139:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e1556e3/0x85ace67feb161f4e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572541.395897:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x7c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572541.396054:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970286 00010000:00010000:1.0:1788572541.396101:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.396116:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.396125:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.396132:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.396139:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.396145:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.396162:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788572541.396420:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572541.396428:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.396436:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572541.396440:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.396447:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.396469:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7c:0x0].0x0 (ffff9627d0ee2a00) 00010000:00010000:0.0:1788572541.396473:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572541.396477:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.396483:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91400/0x85ace67feb161f5c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.396500:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572541.396503:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572541.396505:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572541.396867:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360319232:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 2071us (2378us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.396876:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 948 00010000:00010000:3.0:1788572541.396909:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396919:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396941:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396949:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396957:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396963:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572541.396966:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396974:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396986:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.396992:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000437a71f3/0x85ace67feb161f47 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.397054:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7660050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360318208:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 17172us (30077us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.397061:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1035 00000100:00100000:2.0:1788572541.399493:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360318848:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 19564us (19745us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.399511:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1036 00000100:00100000:1.0:1788572541.403254:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35899:x1875453360319104:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 8144us (8846us total) trans 4294970287 rc 0/0 00000100:00100000:1.0:1788572541.403270:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 869 00000020:00080000:1.0:1788572541.403278:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788572541.403467:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.403597:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.403803:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360319616 00000100:00100000:0.0:1788572541.403812:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360319616 00000100:00100000:0.0:1788572541.403852:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360319872 00000100:00100000:0.0:1788572541.403855:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360319872 00000100:00100000:0.0:1788572541.403878:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3982 00000100:00100000:0.0:1788572541.403885:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35898:x1875453360319616:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.403966:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.403984:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.403997:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.404046:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3983 00000100:00100000:2.0:1788572541.404056:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35897:x1875453360319872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.404088:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404098:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404106:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404254:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404266:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 70 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404274:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404311:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 70 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.404326:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 70 pid: 5838 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.404353:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382346368:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572541.404442:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3900 going to sleep for 11 seconds 00010000:00010000:0.0:1788572541.406468:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.406494:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.406504:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.406546:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.406559:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 71 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.406573:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382346496:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572541.407115:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.411744:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382346496:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.411779:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.412313:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.412350:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.412416:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360320128 00000100:00100000:2.0:1788572541.412424:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360320128 00010000:00010000:2.0:1788572541.412476:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 72 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.412518:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360320000 00000100:00100000:2.0:1788572541.412522:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360320000 00010000:00010000:2.0:1788572541.412536:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 73 pid: 5838 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.412562:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 950 00000100:00100000:2.0:1788572541.412572:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2349:x1875453360320128:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.412588:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572541.412595:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 73 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.412595:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 951 00000100:00100000:3.0:1788572541.412604:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2349:x1875453360320000:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.412606:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 73 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.412613:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572541.412618:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 73 pid: 5838 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412628:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb161e59 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5cb expref: 73 pid: 22536 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.412628:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 73 pid: 5838 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412645:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:3.0:1788572541.412650:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:3.0:1788572541.412656:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412658:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412666:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.412666:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412675:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.412675:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.412693:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572541.412696:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572541.412744:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2349:x1875453360320128:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 173us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.412751:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 950 00010000:00010000:0.0:1788572541.412780:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb161ec9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef5fc expref: 71 pid: 5838 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412780:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.412788:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.412793:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382346368:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572541.412808:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.412814:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.412819:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572541.412985:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970289 00000001:00080000:2.0:1788572541.413193:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970290 00000001:00080000:2.0:1788572541.413243:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970291 00000100:00100000:2.0:1788572541.413351:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382346624:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572541.413544:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2349:x1875453360320000:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 939us (1191us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.413553:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 951 00000100:00100000:2.0:1788572541.413605:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572541.413640:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0cebe40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.415548:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382346624 00000100:00100000:3.0:1788572541.415564:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382346624 00000100:00100000:3.0:1788572541.415633:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1788572541.415758:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382346624:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572541.415882:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382346624:12345-0@lo:1000:mdt00_002.0 Request processed in 128us (2277us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.415945:0:5839:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967560 gen 2 00000004:00080000:2.0:1788572541.415952:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c2734000, transno 4294967556 00000004:00080000:2.0:1788572541.415964:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786ea00, transno 4294967557 00000004:00080000:2.0:1788572541.415967:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786ed80, transno 4294967558 00000004:00080000:2.0:1788572541.415970:0:5839:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967560 committed transno 4294967560 00000100:00100000:2.0:1788572541.415995:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382346624:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1788572541.416803:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970292 00000100:00100000:2.0:1788572541.416893:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382346752:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572541.416938:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572541.416967:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3c00 going to sleep for 11 seconds 00000004:00080000:2.0:1788572541.417195:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 2 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:2.0:1788572541.418087:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382346752 00000100:00100000:2.0:1788572541.418096:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382346752 00000001:00080000:0.0:1788572541.418288:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970288 00000001:00080000:0.0:1788572541.418482:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970294 00000001:00080000:0.0:1788572541.418531:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970295 00000100:00100000:0.0:1788572541.418609:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382346880:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572541.418924:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572541.418969:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628008d8900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.419105:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382346880 00000100:00100000:1.0:1788572541.419111:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382346880 00000100:00100000:1.0:1788572541.419141:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:1.0:1788572541.419150:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:5838:x1875453382346880:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572541.419229:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:5838:x1875453382346880:12345-0@lo:1000:mdt00_001.0 Request processed in 80us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.419237:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:0.0:1788572541.419323:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382346880:0@lo:1000:mdt00_001.0 00000001:00080000:0.0:1788572541.419891:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970296 00000100:00100000:0.0:1788572541.420051:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382347008:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572541.420086:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572541.420107:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.420170:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382347008 00000100:00100000:1.0:1788572541.420175:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382347008 00000100:00100000:1.0:1788572541.420198:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:1.0:1788572541.420206:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:5838:x1875453382347008:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572541.420363:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:5838:x1875453382347008:12345-0@lo:1000:mdt00_001.0 Request processed in 157us (276us total) trans 4294967567 rc 0/0 00000100:00100000:1.0:1788572541.420371:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00100000:3.0:1788572541.420456:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:0.0:1788572541.420882:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382347008:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1788572541.420919:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.420929:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421037:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421043:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.421049:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421063:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421081:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d599a600) 00010000:00010000:0.0:1788572541.421085:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572541.421090:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421096:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb161f71 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x7d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421130:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421136:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421141:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421146:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.421148:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421154:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421161:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:0.0:1788572541.421163:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572541.421170:0:5838:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 158 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421186:0:5838:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 158 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421194:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 158 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.421224:0:5838:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eb400/0x85ace67feb1611a9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef267 expref: 158 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.421236:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382347136:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572541.421306:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.426604:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.426659:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360320256 00000100:00100000:1.0:1788572541.426670:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360320256 00000100:00100000:1.0:1788572541.426736:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 870 00000100:00100000:1.0:1788572541.426747:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e64450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:35867:x1875453360320256:12345-192.168.202.40@tcp:49:diff.0 00000100:00100000:0.0:1788572541.426812:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382347136:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572541.426842:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.426854:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4800/0x85ace67feb15ef89 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.426889:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.426899:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e64450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:35867:x1875453360320256:12345-192.168.202.40@tcp:49:diff.0 Request processed in 153us (303us total) trans 0 rc 0/0 00010000:00010000:0.0:1788572541.426906:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 159 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.426913:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 870 00010000:00010000:0.0:1788572541.426925:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.426964:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 158 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.426980:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 158 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1788572541.426982:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572541.426999:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382347264:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572541.427078:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.429344:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:1.0:1788572541.429366:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382346752:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1788572541.429677:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:5839:x1875453382346752:12345-0@lo:1000:mdt00_002.0 Request processed in 320us (12738us total) trans 4294967568 rc 0/0 00000100:00100000:1.0:1788572541.429685:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:2.0:1788572541.429770:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382346752:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1788572541.429815:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429826:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429835:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429841:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bace00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.429845:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429853:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429892:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d599a000) 00010000:00010000:2.0:1788572541.429896:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572541.429902:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429908:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb161f78 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x7b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429938:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429950:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429956:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429961:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.429964:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429970:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429977:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572541.429979:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572541.429982:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.429987:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb15f038 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430025:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430038:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430048:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430055:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430061:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430066:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430071:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.430074:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430079:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430086:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572541.430088:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.430091:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430096:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161f86 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430104:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430109:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430128:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430133:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.430137:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430143:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430151:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572541.430154:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572541.430156:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.430163:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb161f63 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.430298:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35898:x1875453360319616:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 26410us (26839us total) trans 4294970293 rc 0/0 00000100:00100000:2.0:1788572541.430307:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3982 00000020:00080000:2.0:1788572541.430310:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572541.431965:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.433086:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382347264:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572541.433115:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.434213:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360320384 00000100:00100000:1.0:1788572541.434223:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360320384 00010000:00010000:0.0:1788572541.434368:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 159 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.434447:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 952 00000100:00100000:0.0:1788572541.434467:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:2349:x1875453360320384:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572541.434482:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572541.434489:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 159 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.434501:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 159 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.434523:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb161163 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef252 expref: 159 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.434540:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:0.0:1788572541.434545:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.434556:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.434563:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.434596:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572541.434767:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434776:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434789:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434796:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434803:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434809:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.434814:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434822:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434832:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:3.0:1788572541.434835:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.434839:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434845:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb161f7f lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434858:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434869:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434875:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434880:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.434883:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434889:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434897:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffff9627d599ae00) 00010000:00010000:3.0:1788572541.434899:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572541.434902:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.434907:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161f6a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.435105:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35897:x1875453360319872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 31050us (31506us total) trans 4294970297 rc 0/0 00000100:00100000:3.0:1788572541.435114:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3983 00000020:00080000:3.0:1788572541.435119:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1788572541.435226:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572541.435231:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572541.435263:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 130 count 101 00000040:00080000:3.0:1788572541.435269:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:3.0:1788572541.435378:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382347392:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572541.435428:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572541.435632:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+158:2349:x1875453360320384:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1174us (3670us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.435639:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 952 00000100:00100000:3.0:1788572541.438863:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.438910:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.438973:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360320768 00000100:00100000:2.0:1788572541.438983:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360320768 00000100:00100000:2.0:1788572541.439077:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1037 00000100:00100000:2.0:1788572541.439088:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+31:35867:x1875453360320768:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572541.439107:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788572541.439114:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360320640 00000100:00100000:1.0:1788572541.439120:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360320640 00010000:00010000:2.0:1788572541.439128:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005555114e/0x85ace67feb161f8d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef62d expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.439157:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 871 00002000:00010000:2.0:1788572541.439160:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x46:0x0] initial LVB size: 3, mtime: 0x6a9b7363, atime: 0x6a9b7363, ctime: 0x6a9b736f, blocks: 0xb 00000100:00100000:1.0:1788572541.439168:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35898:x1875453360320640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.439182:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=889s v=(0 0) 00010000:00010000:2.0:1788572541.439188:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.439200:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005555114e/0x85ace67feb161f8d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef62d expref: 32 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.439216:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005555114e/0x85ace67feb161f8d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef62d expref: 32 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.439228:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52c00, rc 0) 00000100:00100000:2.0:1788572541.439285:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+32:35867:x1875453360320768:12345-192.168.202.40@tcp:101:diff.0 Request processed in 199us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.439292:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1037 00000004:00080000:1.0:1788572541.439572:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x7b:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572541.439616:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970298 00010000:00010000:1.0:1788572541.439695:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.439709:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.439720:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.439730:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.439736:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.439743:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.439764:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572541.439834:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35898:x1875453360320640:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 668us (980us total) trans 4294970299 rc 0/0 00000100:00100000:1.0:1788572541.439841:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 871 00000020:00080000:1.0:1788572541.439845:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572541.440306:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572541.440314:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.440321:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572541.440325:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.440333:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.440346:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7b:0x0].0x0 (ffff9627d0ee2100) 00010000:00010000:2.0:1788572541.440349:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572541.440353:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.440359:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161f94 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.440376:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572541.440378:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572541.440380:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572541.446169:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.446287:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382347392:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572541.446427:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321024 00000100:00100000:1.0:1788572541.446437:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321024 00000100:00100000:1.0:1788572541.446527:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 872 00000100:00100000:1.0:1788572541.446540:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35897:x1875453360321024:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572541.446900:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x7d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572541.446944:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970300 00010000:00010000:1.0:1788572541.446987:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.447027:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.447040:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.447049:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.447054:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.447061:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.447079:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572541.447166:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35897:x1875453360321024:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 628us (1006us total) trans 4294970301 rc 0/0 00000100:00100000:1.0:1788572541.447174:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 872 00000020:00080000:1.0:1788572541.447179:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572541.447569:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572541.447577:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.447586:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a90a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572541.447590:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.447600:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.447618:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7d:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:0.0:1788572541.447621:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572541.447626:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 1/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.447633:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161f9b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.447653:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572541.447656:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572541.447658:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572541.450535:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.450575:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.450611:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.450672:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321152 00000100:00100000:1.0:1788572541.450680:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321152 00000100:00100000:1.0:1788572541.450725:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321280 00000100:00100000:1.0:1788572541.450728:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321280 00000100:00100000:1.0:1788572541.450748:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321408 00000100:00100000:1.0:1788572541.450751:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321408 00000100:00100000:1.0:1788572541.450771:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3984 00000100:00100000:1.0:1788572541.450781:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35902:x1875453360321152:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572541.450816:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x87:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572541.450931:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35902:x1875453360321152:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 152us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.450935:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3984 00000100:00100000:1.0:1788572541.450948:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3985 00000100:00100000:1.0:1788572541.450952:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35901:x1875453360321280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572541.450964:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x8b:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572541.450982:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3986 00000100:00100000:1.0:1788572541.450996:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35901:x1875453360321280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 41us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.450998:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35900:x1875453360321408:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.451022:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3985 00000004:00000080:2.0:1788572541.451043:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x86:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572541.451101:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35900:x1875453360321408:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 109us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.451110:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3986 00000100:00100000:3.0:1788572541.452257:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.452425:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321536 00000100:00100000:2.0:1788572541.452433:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321536 00002000:00100000:2.0:1788572541.452486:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x46:0x0] (0->4095) req@ffff9626c1b58380 x1875453360321536/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.452497:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b58380 with x1875453360321536 ext(0->4095) 00002000:00010000:2.0:1788572541.452504:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b58380 00000100:00100000:2.0:1788572541.452537:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 235 00000100:00100000:2.0:1788572541.452547:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+32:2350:x1875453360321536:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572541.452761:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+32:2350:x1875453360321536:12345-192.168.202.40@tcp:3:diff.0 Request processed in 215us (508us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572541.452770:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 235 00002000:00100000:2.0:1788572541.452779:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x46:0x0] (0->4095) req@ffff9626c1b58380 x1875453360321536/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.452787:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b58380 with x1875453360321536 ext(0->4095) 00002000:00010000:2.0:1788572541.452790:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b58380 00000100:00100000:3.0:1788572541.459750:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.459791:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.460066:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321664 00000100:00100000:2.0:1788572541.460077:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321664 00000100:00100000:2.0:1788572541.460136:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360321792 00000100:00100000:2.0:1788572541.460140:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360321792 00000100:00100000:2.0:1788572541.460260:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3987 00000100:00100000:2.0:1788572541.460273:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35901:x1875453360321664:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.460361:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460381:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460404:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460422:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460429:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460435:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460473:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460480:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460487:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572541.460525:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8b:0x0]: returning size 3 00000004:00200000:2.0:1788572541.460529:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572541.460543:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460554:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460561:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460568:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460573:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.460578:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460586:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460598:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572541.460601:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.460606:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460611:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb161fa2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460622:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460627:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460636:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460655:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.460658:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460664:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460671:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xd6032ac1 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572541.460673:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.460676:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460681:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb161fa9 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd6032ac1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460696:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460703:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460709:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460714:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.460716:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460726:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460732:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:2.0:1788572541.460735:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.460738:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460744:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb161fb0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.460825:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35901:x1875453360321664:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 554us (1087us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.460832:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3987 00000100:00100000:2.0:1788572541.460853:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3988 00000100:00100000:2.0:1788572541.460860:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35900:x1875453360321792:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.460886:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460894:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460901:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460913:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460920:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460928:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460944:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460951:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460956:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572541.460975:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x86:0x0]: returning size 3 00000004:00200000:2.0:1788572541.460978:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572541.460985:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460993:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.460998:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461035:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461041:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.461044:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461050:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461059:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572541.461061:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.461064:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461070:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb161fb7 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461079:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461085:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461091:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461095:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.461098:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461104:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461111:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x54c6c725 (ffff9627e7a95500) 00010000:00010000:2.0:1788572541.461113:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.461116:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461122:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb161fbe lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x54c6c725 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461137:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461143:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461148:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461154:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.461156:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461162:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461170:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627ffc16500) 00010000:00010000:2.0:1788572541.461172:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.461175:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.461180:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb161fc5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.461215:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35900:x1875453360321792:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 355us (1422us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.461222:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3988 00000100:00100000:2.0:1788572541.473781:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.473824:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.473883:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360322048 00000100:00100000:1.0:1788572541.473895:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360322048 00000100:00100000:1.0:1788572541.473956:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 873 00000100:00100000:1.0:1788572541.473966:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+158:35867:x1875453360322048:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572541.474373:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:35867:x1875453360322048:12345-192.168.202.40@tcp:35:diff.0 Request processed in 407us (601us total) trans 4294970302 rc 0/0 00000100:00100000:1.0:1788572541.474382:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 873 00000100:00100000:3.0:1788572541.474669:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360322176 00000100:00100000:3.0:1788572541.474675:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360322176 00000100:00100000:3.0:1788572541.474938:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3989 00000100:00100000:3.0:1788572541.474950:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35902:x1875453360322176:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.475030:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475048:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475060:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475076:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475083:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475089:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475115:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/1,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475123:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475129:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.475159:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x87:0x0]: returning size 3 00000004:00200000:3.0:1788572541.475163:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.475173:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475197:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475204:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475210:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475216:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.475220:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475228:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475240:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572541.475243:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.475248:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475254:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb161fcc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475276:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475283:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475290:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475296:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.475299:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475305:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475314:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x31461b99 (ffff9627d599a900) 00010000:00010000:3.0:1788572541.475317:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.475320:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475326:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb161fd3 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x31461b99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475341:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475348:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475354:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475360:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae0a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.475363:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475369:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475379:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627ffc16b00) 00010000:00010000:3.0:1788572541.475382:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.475385:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 2/0,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.475392:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb161fda lrc: 0/0,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.475487:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35902:x1875453360322176:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 539us (1662us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.475495:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3989 00000100:00100000:2.0:1788572541.483385:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.483470:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.483540:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360322432 00000100:00100000:1.0:1788572541.483550:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360322432 00000100:00100000:1.0:1788572541.483605:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360322688 00000100:00100000:1.0:1788572541.483608:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360322688 00000100:00100000:1.0:1788572541.483631:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3990 00000100:00100000:1.0:1788572541.483638:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35900:x1875453360322432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.483659:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.483683:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161fe1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef634 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483789:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483804:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483814:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572541.483824:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x86:0x0]: returning size 3 00000100:00100000:3.0:1788572541.483825:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3991 00000100:00100000:3.0:1788572541.483837:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35901:x1875453360322688:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.483848:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.483861:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb161ff6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef63b expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483908:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483915:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483920:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483924:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c9a91c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.483924:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483930:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483945:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.483945:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483964:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627ffc16500) 00010000:00010000:3.0:1788572541.483965:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.483969:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572541.483974:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.483978:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8b:0x0]: returning size 3 00010000:00010000:1.0:1788572541.483980:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb161fe8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.484044:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161fe1 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef634 expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.484053:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572541.484055:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572541.484058:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb161fe1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef634 expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484060:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.484068:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a90a00, rc 0) 00010000:00010000:3.0:1788572541.484071:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484079:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484086:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b91000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.484090:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484097:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484106:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:3.0:1788572541.484109:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572541.484113:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484124:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b91000/0x85ace67feb161ffd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.484136:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35900:x1875453360322432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 498us (761us total) trans 4294970303 rc 301/301 00000100:00100000:1.0:1788572541.484142:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3990 00010000:00010000:3.0:1788572541.484153:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb161ff6 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef63b expref: 83 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484164:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572541.484166:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572541.484170:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb161ff6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef63b expref: 83 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.484252:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ae1600, rc 0) 00000100:00100000:3.0:1788572541.484294:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35901:x1875453360322688:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 460us (833us total) trans 4294970304 rc 301/301 00000100:00100000:3.0:1788572541.484301:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3991 00000100:00100000:0.0:1788572541.489482:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.489647:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.489781:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360322816 00000100:00100000:1.0:1788572541.489793:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360322816 00000100:00100000:1.0:1788572541.489888:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323072 00000100:00100000:1.0:1788572541.489892:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323072 00000100:00100000:0.0:1788572541.489928:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3992 00000100:00100000:0.0:1788572541.489941:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+159:35867:x1875453360322816:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572541.489965:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.489987:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16200b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef642 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490092:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490110:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490123:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490139:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490146:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490153:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490749:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb162020 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490760:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb162020 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490769:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb162020 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.490812:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb9:0x0]: returning size 3 00000004:00200000:0.0:1788572541.490817:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.490844:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb162020 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490859:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb162020 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490868:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490875:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490882:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490888:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bacc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.490893:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490901:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490918:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572541.490920:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.490926:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490932:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162012 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490946:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490952:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490957:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490962:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.490965:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490970:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490977:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x6e9b5ca3 (ffff9627e7451700) 00010000:00010000:0.0:1788572541.490980:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.490983:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.490989:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162019 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491065:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16200b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef642 expref: 161 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491080:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad400/0x85ace67feb162020 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef642 expref: 160 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491086:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627e7451300) 00010000:00010000:0.0:1788572541.491089:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.491091:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad400, rc 0) 00000100:00100000:0.0:1788572541.491359:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+160:35867:x1875453360322816:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1419us (1971us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.491369:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3992 00000100:00100000:0.0:1788572541.491412:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3993 00000100:00100000:0.0:1788572541.491423:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35902:x1875453360323072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.491436:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.491448:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162027 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef649 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491525:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491534:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491540:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.491552:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x87:0x0]: returning size 3 00010000:00010000:0.0:1788572541.491643:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491651:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491657:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491662:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.491666:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491672:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491680:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627ffc16b00) 00010000:00010000:0.0:1788572541.491683:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572541.491686:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491691:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16202e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491722:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162027 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef649 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491732:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572541.491734:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572541.491739:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162027 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef649 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.491748:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:0.0:1788572541.491774:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35902:x1875453360323072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 355us (2128us total) trans 4294970305 rc 301/301 00000100:00100000:0.0:1788572541.491781:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3993 00000100:00100000:0.0:1788572541.492798:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.492834:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.493283:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323200 00000100:00100000:1.0:1788572541.493292:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323200 00000100:00100000:1.0:1788572541.493338:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323328 00000100:00100000:1.0:1788572541.493342:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323328 00000100:00100000:1.0:1788572541.493435:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3994 00000100:00100000:1.0:1788572541.493445:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35901:x1875453360323200:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572541.493485:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x8b:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572541.493582:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35901:x1875453360323200:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 138us (789us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.493589:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3994 00000100:00100000:1.0:1788572541.493678:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3995 00000100:00100000:1.0:1788572541.493686:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35900:x1875453360323328:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572541.493704:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x86:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572541.493752:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35900:x1875453360323328:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 67us (916us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.493758:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3995 00000100:00100000:0.0:1788572541.502300:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.502477:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323456 00000100:00100000:2.0:1788572541.502488:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323456 00000100:00100000:2.0:1788572541.502548:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3996 00000100:00100000:2.0:1788572541.502559:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35901:x1875453360323456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.502581:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.502604:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16203c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef650 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.502682:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.502699:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.502711:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.502727:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.502734:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.502741:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.504747:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.504803:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.504893:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323712 00000100:00100000:3.0:1788572541.504904:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323712 00000100:00100000:3.0:1788572541.504976:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1038 00000100:00100000:3.0:1788572541.504985:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+32:2347:x1875453360323712:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572541.505032:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.505054:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd049a5e/0x85ace67feb162051 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef657 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572541.505080:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x45:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572541.505099:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=889s v=(0 0) 00010000:00010000:3.0:1788572541.505105:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572541.505115:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd049a5e/0x85ace67feb162051 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef657 expref: 33 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.505132:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd049a5e/0x85ace67feb162051 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef657 expref: 33 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.505145:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b91000, rc 0) 00000100:00100000:3.0:1788572541.505197:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+33:2347:x1875453360323712:12345-192.168.202.40@tcp:101:diff.0 Request processed in 214us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.505203:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1038 00000100:00100000:2.0:1788572541.505322:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323584 00000100:00100000:2.0:1788572541.505327:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323584 00000100:00100000:2.0:1788572541.505366:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3997 00000100:00100000:2.0:1788572541.505374:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35902:x1875453360323584:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572541.505403:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x87:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572541.505502:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35902:x1875453360323584:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 129us (765us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.505508:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3997 00010000:00010000:0.0:1788572541.511151:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162058 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511176:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162058 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511190:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162058 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.511228:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572541.511233:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.511256:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162058 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511270:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511277:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511284:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511291:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.511296:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511304:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511318:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572541.511321:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.511326:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511333:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162043 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511346:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511352:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511358:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511362:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.511364:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511370:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511381:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627e7a95500) 00010000:00010000:0.0:1788572541.511384:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.511387:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511393:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb16204a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511437:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb16203c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef650 expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511453:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162058 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef650 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.511473:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572541.511476:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572541.511479:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:0.0:1788572541.511559:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35901:x1875453360323456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9001us (9269us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.511569:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3996 00000100:00100000:1.0:1788572541.518721:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.518765:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.518792:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.518824:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360323968 00000100:00100000:1.0:1788572541.518833:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360323968 00000100:00100000:1.0:1788572541.518883:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360324352 00000100:00100000:1.0:1788572541.518887:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360324352 00000100:00100000:1.0:1788572541.518903:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360324224 00000100:00100000:1.0:1788572541.518905:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360324224 00000100:00100000:1.0:1788572541.518924:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3998 00000100:00100000:1.0:1788572541.518941:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+160:35867:x1875453360323968:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572541.518961:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.518983:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16205f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef65e expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.519117:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb162066 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.519132:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb162066 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.519145:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb162066 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572541.519156:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb9:0x0]: returning size 3 00000100:00100000:0.0:1788572541.519320:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 3999 00000100:00100000:0.0:1788572541.519331:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35900:x1875453360324352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.519341:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.519352:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162074 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef66c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519395:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519406:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519414:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519428:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519435:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519443:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519765:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162089 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519775:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162089 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519783:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162089 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.519802:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572541.519805:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.519823:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162089 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519836:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519849:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519856:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519862:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.519866:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519874:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519887:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572541.519890:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.519896:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519903:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16207b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519917:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519923:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519929:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519934:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.519937:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519943:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519950:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627e7451700) 00010000:00010000:0.0:1788572541.519953:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.519955:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519961:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162082 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.519996:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162074 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef66c expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520044:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162089 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef66c expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520052:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572541.520055:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572541.520057:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bacc00, rc 0) 00000100:00100000:0.0:1788572541.520118:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35900:x1875453360324352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 788us (1351us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.520125:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3999 00000100:00100000:0.0:1788572541.520245:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4000 00000100:00100000:0.0:1788572541.520254:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35902:x1875453360324224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.520263:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.520266:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572541.520278:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162058 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef650 expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520289:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572541.520293:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572541.520300:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb162090 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef665 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572541.520703:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x46:0x0], index 1: 0 00010000:00010000:0.0:1788572541.520892:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb162097 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520904:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb162097 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520913:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb162097 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520960:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb162090 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef665 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520975:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb162097 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef665 expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.520982:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7451400) 00010000:00010000:0.0:1788572541.520985:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.520987:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee600, rc 0) 00000100:00100000:0.0:1788572541.521065:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35902:x1875453360324224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 811us (2270us total) trans 4294970306 rc 0/0 00000100:00100000:0.0:1788572541.521076:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4000 00010000:00010000:1.0:1788572541.521350:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16205f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef65e expref: 163 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.521364:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb162066 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef65e expref: 162 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.521372:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627e7451300) 00010000:00010000:1.0:1788572541.521374:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572541.521377:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246d800, rc 0) 00000100:00100000:1.0:1788572541.521413:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:35867:x1875453360323968:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2481us (2704us total) trans 4294970307 rc 0/0 00000100:00100000:1.0:1788572541.521419:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 3998 00000100:00100000:0.0:1788572541.525946:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.526074:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360324608 00000100:00100000:2.0:1788572541.526085:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360324608 00000100:00100000:2.0:1788572541.526143:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4001 00000100:00100000:2.0:1788572541.526153:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35901:x1875453360324608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.526178:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.526198:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1620a5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef673 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572541.526808:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x47:0x0], index 0: 0 00010000:00010000:3.0:1788572541.527023:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb1620ac lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.527039:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb1620ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.527053:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb1620ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.527115:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb1620a5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef673 expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.527136:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb1620ac lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef673 expref: 88 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.527145:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a300) 00010000:00010000:3.0:1788572541.527148:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.527152:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90600, rc 0) 00000100:00100000:3.0:1788572541.527216:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35901:x1875453360324608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1063us (1279us total) trans 4294970308 rc 0/0 00000100:00100000:3.0:1788572541.527224:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4001 00000100:00100000:1.0:1788572541.527382:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.529493:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360324736 00000100:00100000:2.0:1788572541.529509:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360324736 00002000:00100000:2.0:1788572541.529569:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x45:0x0] (0->4095) req@ffff962801f90e00 x1875453360324736/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.529580:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f90e00 with x1875453360324736 ext(0->4095) 00002000:00010000:2.0:1788572541.529585:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff962801f90e00 00000100:00100000:2.0:1788572541.529627:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 236 00000100:00100000:2.0:1788572541.529652:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+33:2350:x1875453360324736:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572541.529875:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+33:2350:x1875453360324736:12345-192.168.202.40@tcp:3:diff.0 Request processed in 240us (2494us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572541.529884:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 236 00002000:00100000:2.0:1788572541.529891:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x45:0x0] (0->4095) req@ffff962801f90e00 x1875453360324736/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.529905:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f90e00 with x1875453360324736 ext(0->4095) 00002000:00010000:2.0:1788572541.529908:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff962801f90e00 00000100:00100000:1.0:1788572541.530573:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.530649:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360324864 00000100:00100000:2.0:1788572541.530656:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360324864 00000100:00100000:2.0:1788572541.530710:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 298 00000100:00100000:2.0:1788572541.530720:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:2348:x1875453360324864:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.530734:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572541.530763:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb162066 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef65e expref: 162 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.530782:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffff9627e7451300) 00010000:00010000:2.0:1788572541.530787:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572541.530825:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+161:2348:x1875453360324864:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 107us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.530833:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 298 00000100:00100000:1.0:1788572541.531843:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.531937:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360325120 00000100:00100000:2.0:1788572541.531944:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360325120 00000100:00100000:2.0:1788572541.531997:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4002 00000100:00100000:2.0:1788572541.532035:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35900:x1875453360325120:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.532058:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.532061:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572541.532080:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bacc00/0x85ace67feb162089 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef66c expref: 88 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.532095:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572541.532100:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572541.532114:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1620ba lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef67a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572541.532806:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x47:0x0], index 1: 0 00010000:00010000:2.0:1788572541.533059:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1620c1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.533075:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1620c1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.533086:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1620c1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.533148:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1620ba lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef67a expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.533164:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1620c1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef67a expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.533172:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572541.533175:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572541.533177:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:2.0:1788572541.533228:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35900:x1875453360325120:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1223us (1390us total) trans 4294970309 rc 0/0 00000100:00100000:2.0:1788572541.533236:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4002 00000100:00100000:1.0:1788572541.534285:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.534428:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360325376 00000100:00100000:3.0:1788572541.534437:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360325376 00000100:00100000:3.0:1788572541.534497:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 874 00000100:00100000:3.0:1788572541.534506:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+161:35867:x1875453360325376:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572541.534792:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:35867:x1875453360325376:12345-192.168.202.40@tcp:35:diff.0 Request processed in 286us (512us total) trans 4294970310 rc 0/0 00000100:00100000:3.0:1788572541.534800:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 874 00000100:00100000:2.0:1788572541.540288:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.540368:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360325504 00000100:00100000:3.0:1788572541.540379:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360325504 00000100:00100000:3.0:1788572541.540440:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4003 00000100:00100000:3.0:1788572541.540451:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:35867:x1875453360325504:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572541.540485:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.540507:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1620cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef681 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540577:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540593:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540606:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540624:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540631:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540638:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540954:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb1620e4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540964:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb1620e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.540971:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb1620e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.541041:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc1:0x0]: returning size 3 00000004:00200000:3.0:1788572541.541046:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.541077:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb1620e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541089:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb1620e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541097:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541104:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541115:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541120:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.541124:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541132:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541143:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.541146:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.541152:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541158:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1620d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541169:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541174:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541180:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541185:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.541188:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541198:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541205:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xe44515a3 (ffff9627d599a100) 00010000:00010000:3.0:1788572541.541207:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.541210:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541214:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1620dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541243:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1620cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef681 expref: 164 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541256:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae0a00/0x85ace67feb1620e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef681 expref: 163 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541263:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627d599a200) 00010000:00010000:3.0:1788572541.541265:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.541267:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ae0a00, rc 0) 00000100:00100000:1.0:1788572541.541276:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.541301:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.541314:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+163:35867:x1875453360325504:12345-192.168.202.40@tcp:101:diff.0 Request processed in 865us (1038us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.541322:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4003 00000100:00100000:3.0:1788572541.541339:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360325632 00000100:00100000:3.0:1788572541.541343:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360325632 00000100:00100000:3.0:1788572541.541368:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4004 00000100:00100000:3.0:1788572541.541376:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35900:x1875453360325632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.541391:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.541401:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1620eb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef688 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541438:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541446:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541454:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541480:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541487:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541494:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541670:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541681:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541688:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.541706:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:3.0:1788572541.541709:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.541721:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541730:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541736:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541742:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541747:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.541750:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541757:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541766:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572541.541768:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.541771:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541777:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1620f2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541786:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541792:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541802:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541808:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.541811:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541817:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541823:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627d599a500) 00010000:00010000:3.0:1788572541.541825:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.541828:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541833:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050600/0x85ace67feb1620f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541855:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1620eb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541865:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.541871:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572541.541874:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572541.541876:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050400, rc 0) 00000100:00100000:3.0:1788572541.541924:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35900:x1875453360325632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 548us (647us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.541933:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4004 00000100:00100000:3.0:1788572541.541945:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360325888 00000100:00100000:3.0:1788572541.541949:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360325888 00000100:00100000:3.0:1788572541.541973:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4005 00000100:00100000:3.0:1788572541.541981:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35902:x1875453360325888:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.542033:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572541.542047:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb162097 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef665 expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542057:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7451400) 00010000:00010000:3.0:1788572541.542066:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572541.542079:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542087:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 162 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542095:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542101:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542132:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542145:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 89 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.542160:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ca8c9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382347520:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572541.542235:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 162 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.542244:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 162 pid: 18263 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.542253:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ca8cb100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382347648:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572541.542276:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.555447:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.556961:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360326016 00000100:00100000:3.0:1788572541.556975:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360326016 00000100:00100000:3.0:1788572541.557146:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1039 00000100:00100000:3.0:1788572541.557157:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2350:x1875453360326016:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572541.557178:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.557200:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000065bce4ff/0x85ace67feb16210e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef68f expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572541.557228:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x43:0x0] initial LVB size: 3, mtime: 0x6a9b7363, atime: 0x6a9b7363, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572541.557254:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=889s v=(0 0) 00010000:00010000:3.0:1788572541.557259:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572541.557269:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000065bce4ff/0x85ace67feb16210e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef68f expref: 43 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.557286:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000065bce4ff/0x85ace67feb16210e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef68f expref: 43 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.557303:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050600, rc 0) 00000100:00100000:3.0:1788572541.557378:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2350:x1875453360326016:12345-192.168.202.40@tcp:101:diff.0 Request processed in 222us (1942us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.557385:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1039 00000100:00100000:1.0:1788572541.562350:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.562442:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360326144 00000100:00100000:2.0:1788572541.562453:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360326144 00010000:00010000:2.0:1788572541.562506:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 163 pid: 18263 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.562547:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 953 00000100:00100000:2.0:1788572541.562558:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e52850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+163:2350:x1875453360326144:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.562572:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572541.562579:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 163 pid: 18263 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.562591:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 163 pid: 18263 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.562605:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572541.562615:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572541.562621:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.562624:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360326272 00000100:00100000:1.0:1788572541.562631:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360326272 00010000:00010000:2.0:1788572541.562633:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572541.562652:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 90 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.562678:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 954 00000100:00100000:1.0:1788572541.562686:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:2348:x1875453360326272:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.562695:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.562700:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 90 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.562712:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e52850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+163:2350:x1875453360326144:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 155us (371us total) trans 0 rc 0/0 00010000:00010000:1.0:1788572541.562722:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 90 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.562723:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 953 00010000:00010000:1.0:1788572541.562743:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572541.562747:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.562755:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.562764:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.562785:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.562810:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:2348:x1875453360326272:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 125us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.562816:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 954 00010000:00010000:3.0:1788572541.562835:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162100 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef688 expref: 89 pid: 9040 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.562853:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ca8c9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382347520:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572541.562868:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb15da2e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddf3 expref: 162 pid: 18263 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.562877:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ca8cb100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382347648:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572541.562896:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.562906:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.562912:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.562934:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.562944:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.562956:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563245:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563251:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563259:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563264:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.563269:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563277:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563287:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572541.563290:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.563293:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563299:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb162107 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563308:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563313:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563319:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563323:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.563326:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563332:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563338:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xc42344bc (ffff9627d599aa00) 00010000:00010000:3.0:1788572541.563341:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.563344:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.563350:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162115 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xc42344bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.563415:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35902:x1875453360325888:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 21433us (22111us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572541.563423:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4005 00000100:00100000:2.0:1788572541.567680:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.567837:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.568337:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360326528 00000100:00100000:0.0:1788572541.568349:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360326528 00000100:00100000:0.0:1788572541.568401:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360326656 00000100:00100000:0.0:1788572541.568404:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360326656 00000100:00100000:0.0:1788572541.568427:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4006 00000100:00100000:0.0:1788572541.568437:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+162:35867:x1875453360326528:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572541.568476:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.568500:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16211c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef696 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.568688:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb162123 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.568704:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb162123 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.568783:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb162123 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.568797:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc1:0x0]: returning size 3 00010000:00010000:0.0:1788572541.569389:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16211c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef696 expref: 165 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.569410:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb162123 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef696 expref: 164 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.569427:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627d599a200) 00010000:00010000:0.0:1788572541.569430:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.569434:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc 0) 00000100:00100000:0.0:1788572541.569521:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:35867:x1875453360326528:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1084us (1853us total) trans 4294970311 rc 0/0 00000100:00100000:0.0:1788572541.569529:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4006 00000100:00100000:2.0:1788572541.569650:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4007 00000100:00100000:2.0:1788572541.569663:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35902:x1875453360326656:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.569675:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.569690:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162131 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef69d expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.569795:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.569806:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.569816:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.570262:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162131 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 90 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.570278:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.570285:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:2.0:1788572541.570288:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572541.570290:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:2.0:1788572541.570325:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35902:x1875453360326656:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 667us (2489us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.570331:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4007 00000100:00100000:1.0:1788572541.571197:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.571330:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360326912 00000100:00100000:1.0:1788572541.571337:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360326912 00000100:00100000:0.0:1788572541.581590:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4008 00000100:00100000:0.0:1788572541.581618:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35901:x1875453360326912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.581683:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572541.581713:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb1620ac lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef673 expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.581864:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a300) 00010000:00010000:0.0:1788572541.581877:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572541.581905:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.581920:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.581941:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582083:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582092:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582099:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582447:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582455:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582476:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582482:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c2477200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.582487:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582495:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582506:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572541.582509:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.582512:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582519:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb16213f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582528:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582606:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582614:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582619:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.582622:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582629:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582637:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x6f3d8f0 (ffff9627e7451700) 00010000:00010000:0.0:1788572541.582639:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.582643:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.582648:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162146 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x6f3d8f0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.582845:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35901:x1875453360326912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1238us (11648us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572541.582853:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4008 00000100:00100000:3.0:1788572541.589130:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.589173:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.589209:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.589248:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360327424 00000100:00100000:1.0:1788572541.589262:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360327424 00000100:00100000:1.0:1788572541.589333:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 299 00000100:00100000:1.0:1788572541.589343:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:2350:x1875453360327424:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.589360:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.589385:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb162123 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef696 expref: 164 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.589405:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffff9627d599a200) 00010000:00010000:1.0:1788572541.589411:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.589480:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+163:2350:x1875453360327424:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 138us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.589489:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 299 00000100:00100000:0.0:1788572541.589557:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360327296 00000100:00100000:0.0:1788572541.589562:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360327296 00002000:00100000:0.0:1788572541.589598:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x43:0x0] (0->4095) req@ffff9627ca8caa00 x1875453360327296/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572541.589608:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8caa00 with x1875453360327296 ext(0->4095) 00002000:00010000:0.0:1788572541.589613:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ca8caa00 00000100:00100000:0.0:1788572541.589641:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 237 00000100:00100000:0.0:1788572541.589649:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2350:x1875453360327296:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572541.589833:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2350:x1875453360327296:12345-192.168.202.40@tcp:3:diff.0 Request processed in 184us (623us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572541.589841:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 237 00002000:00100000:0.0:1788572541.589857:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x43:0x0] (0->4095) req@ffff9627ca8caa00 x1875453360327296/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572541.589866:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8caa00 with x1875453360327296 ext(0->4095) 00002000:00010000:0.0:1788572541.589869:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ca8caa00 00000100:00100000:0.0:1788572541.589889:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360327168 00000100:00100000:0.0:1788572541.589893:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360327168 00000100:00100000:0.0:1788572541.589929:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4009 00000100:00100000:0.0:1788572541.589936:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35902:x1875453360327168:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.590048:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.590064:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.590075:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.590426:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.590501:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.590525:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360327552 00000100:00100000:3.0:1788572541.590530:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360327552 00000100:00100000:3.0:1788572541.590557:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360327808 00000100:00100000:3.0:1788572541.590561:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360327808 00000100:00100000:3.0:1788572541.590581:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4010 00000100:00100000:3.0:1788572541.590590:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35901:x1875453360327552:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.590602:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.590620:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162154 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6a4 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.590806:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4011 00000100:00100000:1.0:1788572541.590816:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35900:x1875453360327808:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.590846:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.590863:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1620c1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef67a expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.590877:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:1.0:1788572541.590886:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000040:00080000:0.0:1788572541.590889:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].56, 88 off12712 00000004:00080000:0.0:1788572541.590904:0:22536:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 889659898698ns, next at 890659891590ns, rc = 0 00010000:00010000:1.0:1788572541.590904:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.590918:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.590928:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.590942:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.590949:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.590956:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591101:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591109:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591117:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591123:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.591129:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591136:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572541.591140:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000019eb15b9 reply state 000000004460f5d1 transno 4294970312 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591147:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572541.591154:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.591159:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591165:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16215b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591175:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591181:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591188:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591193:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.591196:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591203:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591209:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xbda4cba6 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572541.591211:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.591214:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.591220:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162162 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xbda4cba6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.591227:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572541.591283:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35900:x1875453360327808:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 467us (780us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572541.591293:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35902:x1875453360327168:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1354us (2171us total) trans 4294970312 rc 0/0 00000100:00100000:1.0:1788572541.591293:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4011 00000100:00100000:0.0:1788572541.591304:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4009 00010000:00010000:3.0:1788572541.591512:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.591524:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.591533:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.591600:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162154 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.591616:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 90 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.591624:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:3.0:1788572541.591627:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572541.591629:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc 0) 00000100:00100000:3.0:1788572541.591854:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35901:x1875453360327552:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1264us (1428us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.591861:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4010 00010000:00010000:1.0:1788572541.594961:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.594980:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.594988:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.594993:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.594999:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.595046:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.595064:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:1.0:1788572541.595067:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572541.595071:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.595078:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16214d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.596798:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.596836:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.596931:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.601204:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360328064 00000100:00100000:2.0:1788572541.601220:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360328064 00000100:00100000:0.0:1788572541.601434:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360328192 00000100:00100000:0.0:1788572541.601440:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360328192 00000100:00100000:3.0:1788572541.601497:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360328320 00000100:00100000:3.0:1788572541.601504:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360328320 00000100:00100000:2.0:1788572541.601529:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4012 00000100:00100000:2.0:1788572541.601551:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35901:x1875453360328064:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.601666:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.601689:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.601702:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.601782:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4013 00000100:00100000:3.0:1788572541.601792:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35900:x1875453360328320:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.601802:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572541.601811:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.601833:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162177 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6b2 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.601854:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360328576 00000100:00100000:3.0:1788572541.601860:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4014 00000100:00100000:0.0:1788572541.601862:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360328576 00000100:00100000:3.0:1788572541.601870:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35902:x1875453360328192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.601879:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.601891:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6ab expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.601903:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 875 00000100:00100000:0.0:1788572541.601913:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+163:35867:x1875453360328576:12345-192.168.202.40@tcp:35:diff.0 00010000:00010000:3.0:1788572541.601969:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.601984:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.602041:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:3.0:1788572541.602045:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572541.602049:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050c00, rc 0) 00000100:00100000:3.0:1788572541.602109:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35902:x1875453360328192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 241us (5271us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.602116:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4014 00000040:00080000:2.0:1788572541.602197:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].59, 88 off12904 00000004:00080000:2.0:1788572541.602206:0:5837:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 889671201635ns, next at 890671198982ns, rc = 0 00010000:00010000:3.0:1788572541.602274:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572541.602283:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a8a89f65 reply state 000000001996d7c7 transno 4294970314 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.602289:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.602297:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572541.602303:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.602362:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35901:x1875453360328064:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 814us (5573us total) trans 4294970314 rc 0/0 00000100:00100000:2.0:1788572541.602368:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4012 00010000:00010000:3.0:1788572541.603569:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051c00/0x85ace67feb162177 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.603591:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.603599:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:3.0:1788572541.603602:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572541.603605:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051800, rc 0) 00000100:00100000:3.0:1788572541.603793:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35900:x1875453360328320:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2001us (6860us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.603802:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4013 00000100:00100000:0.0:1788572541.605321:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:35867:x1875453360328576:12345-192.168.202.40@tcp:35:diff.0 Request processed in 3401us (3510us total) trans 4294970313 rc 0/0 00000100:00100000:0.0:1788572541.605333:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 875 00010000:00010000:0.0:1788572541.607364:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.607379:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.607386:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.607391:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.607397:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.607405:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.607422:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572541.607425:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572541.607431:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.607436:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162170 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.609519:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.609604:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.609616:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360328704 00000100:00100000:2.0:1788572541.609625:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360328704 00000100:00100000:2.0:1788572541.609758:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1040 00000100:00100000:2.0:1788572541.609770:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360328704:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.610164:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360328704:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 395us (654us total) trans 4294967566 rc 0/0 00000100:00100000:2.0:1788572541.610185:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1040 00000100:00100000:3.0:1788572541.610285:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360328832 00000100:00100000:3.0:1788572541.610291:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360328832 00000100:00100000:3.0:1788572541.610348:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4015 00000100:00100000:3.0:1788572541.610359:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35901:x1875453360328832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.610381:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.610549:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6b9 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.610613:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.610630:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.611860:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:3.0:1788572541.611868:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572541.611873:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050400, rc 0) 00000100:00100000:3.0:1788572541.611961:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35901:x1875453360328832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1600us (2351us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.611970:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4015 00000100:00100000:1.0:1788572541.613252:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.613288:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.613520:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360329088 00000100:00100000:2.0:1788572541.613529:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360329088 00000100:00100000:2.0:1788572541.613588:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360329216 00000100:00100000:2.0:1788572541.613592:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360329216 00000100:00100000:2.0:1788572541.613622:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4016 00000100:00100000:2.0:1788572541.613630:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35900:x1875453360329088:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.613722:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4017 00000100:00100000:1.0:1788572541.613732:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:35867:x1875453360329216:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572541.613751:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.613772:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162193 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6c0 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.613846:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.613861:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.613874:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.613890:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.613898:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.613906:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.614256:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.614267:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.614275:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.614280:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1621af lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.614287:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1621af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.614293:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1621af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572541.614321:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc0:0x0]: returning size 3 00000004:00200000:1.0:1788572541.614325:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000040:00080000:2.0:1788572541.614619:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].57, 88 off12800 00000004:00010000:0.0:1788572541.614925:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1621af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.614941:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1621af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.614950:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.614969:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.614978:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.614984:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.614989:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615000:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615038:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572541.615042:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.615048:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615056:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb16219a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615072:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615078:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615084:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615090:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.615093:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615099:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615107:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x2c9ecea3 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572541.615109:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572541.615112:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615120:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1621a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572541.615159:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000213a2ddf reply state 000000009d629d58 transno 4294970315 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.615171:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788572541.615177:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162193 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6c0 expref: 166 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615202:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bace00/0x85ace67feb1621af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6c0 expref: 165 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.615215:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627d0ee2700) 00010000:00010000:0.0:1788572541.615217:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.615219:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bace00, rc 0) 00000100:00100000:0.0:1788572541.615290:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:35867:x1875453360329216:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1555us (1995us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.615290:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35900:x1875453360329088:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1655us (2039us total) trans 4294970315 rc 0/0 00000100:00100000:0.0:1788572541.615308:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4017 00000100:00100000:2.0:1788572541.615316:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4016 00010000:00010000:1.0:1788572541.622449:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.622479:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.622489:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.622497:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.622502:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.622511:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.622530:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:1.0:1788572541.622534:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572541.622539:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.622544:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1621a8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.626076:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.626371:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.626410:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.626475:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.626501:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360329600 00000100:00100000:1.0:1788572541.626510:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360329600 00000100:00100000:1.0:1788572541.626556:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360329856 00000100:00100000:1.0:1788572541.626560:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360329856 00000100:00100000:1.0:1788572541.626581:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1041 00000100:00100000:1.0:1788572541.626591:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+33:2349:x1875453360329600:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572541.626609:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.626630:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a608c202/0x85ace67feb1621b6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6c7 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572541.626672:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x43:0x0] initial LVB size: 3, mtime: 0x6a9b7363, atime: 0x6a9b7363, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:1.0:1788572541.626691:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=889s v=(0 0) 00010000:00010000:1.0:1788572541.626696:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x43:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.626705:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a608c202/0x85ace67feb1621b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6c7 expref: 34 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.626722:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a608c202/0x85ace67feb1621b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x43:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6c7 expref: 34 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.626736:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bacc00, rc 0) 00000100:00100000:1.0:1788572541.626783:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+34:2349:x1875453360329600:12345-192.168.202.40@tcp:101:diff.0 Request processed in 194us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.626791:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1041 00000100:00100000:1.0:1788572541.626809:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1042 00000100:00100000:1.0:1788572541.626816:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360329856:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.626978:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360329472 00000100:00100000:3.0:1788572541.626984:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360329472 00000100:00100000:3.0:1788572541.627027:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 876 00000100:00100000:3.0:1788572541.627034:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e65850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35902:x1875453360329472:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.627063:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360329856:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 248us (587us total) trans 4294967569 rc 0/0 00000100:00100000:1.0:1788572541.627070:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1042 00000100:00100000:3.0:1788572541.627350:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e65850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35902:x1875453360329472:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 315us (1283us total) trans 4294970316 rc 0/0 00000100:00100000:3.0:1788572541.627357:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 876 00000100:00100000:0.0:1788572541.627383:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360329728 00000100:00100000:0.0:1788572541.627388:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360329728 00000100:00100000:0.0:1788572541.630793:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4018 00000100:00100000:0.0:1788572541.630820:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35900:x1875453360329728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.630846:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.630873:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6ce expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.630954:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.630969:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.630994:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:0.0:1788572541.630998:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572541.631020:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:0.0:1788572541.636069:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35900:x1875453360329728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5247us (9683us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.636088:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4018 00000100:00100000:0.0:1788572541.640829:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.640829:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.640944:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.641123:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360330112 00000100:00100000:0.0:1788572541.641134:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360330112 00000100:00100000:0.0:1788572541.641188:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360330624 00000100:00100000:0.0:1788572541.641192:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360330624 00000100:00100000:0.0:1788572541.641215:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4019 00000100:00100000:0.0:1788572541.641224:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:35867:x1875453360330112:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572541.641248:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1788572541.641262:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4020 00010000:00010000:0.0:1788572541.641270:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1621c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6d5 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.641273:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35902:x1875453360330624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.641384:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35902:x1875453360330624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 112us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.641392:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4020 00010000:00010000:0.0:1788572541.641669:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1621cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.641685:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1621cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.641697:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1621cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.641710:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc0:0x0]: returning size 3 00010000:00010000:0.0:1788572541.641841:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1621c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6d5 expref: 168 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.641863:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1621cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6d5 expref: 167 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.641873:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627d0ee2700) 00010000:00010000:0.0:1788572541.641877:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.641882:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:0.0:1788572541.641922:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360330112:12345-192.168.202.40@tcp:101:diff.0 Request processed in 699us (1103us total) trans 4294970317 rc 0/0 00000100:00100000:0.0:1788572541.641929:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4019 00000100:00100000:3.0:1788572541.644804:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360330368 00000100:00100000:3.0:1788572541.644821:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360330368 00000100:00100000:3.0:1788572541.644928:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 877 00000100:00100000:3.0:1788572541.644939:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35901:x1875453360330368:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.646847:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35901:x1875453360330368:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1898us (6006us total) trans 4294970318 rc 0/0 00000100:00100000:3.0:1788572541.646862:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 877 00000100:00100000:2.0:1788572541.649843:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.649977:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360330752 00000100:00100000:1.0:1788572541.649988:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360330752 00000100:00100000:2.0:1788572541.650210:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1043 00000100:00100000:2.0:1788572541.650223:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360330752:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.650561:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360330752:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 340us (726us total) trans 4294967567 rc 0/0 00000100:00100000:2.0:1788572541.650572:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1043 00000100:00100000:1.0:1788572541.651232:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.651270:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.651294:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331136 00000100:00100000:1.0:1788572541.651300:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331136 00000100:00100000:1.0:1788572541.651345:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 300 00000100:00100000:1.0:1788572541.651354:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:2347:x1875453360331136:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.651368:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1788572541.651487:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331008 00000100:00100000:3.0:1788572541.651492:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331008 00000100:00100000:3.0:1788572541.651533:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4021 00000100:00100000:3.0:1788572541.651540:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35901:x1875453360331008:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.651612:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35901:x1875453360331008:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 72us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.651619:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4021 00010000:00010000:1.0:1788572541.651719:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1621cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6d5 expref: 167 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.651743:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffff9627d0ee2700) 00010000:00010000:1.0:1788572541.651749:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.651770:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:2347:x1875453360331136:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 417us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.651776:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 300 00000100:00100000:1.0:1788572541.656334:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.656440:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.656552:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331264 00000100:00100000:2.0:1788572541.656564:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331264 00000100:00100000:2.0:1788572541.656632:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4022 00000100:00100000:2.0:1788572541.656653:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35902:x1875453360331264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.656685:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.656712:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1621d9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6dc expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.656807:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1621e0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.656825:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1621e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.656839:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1621e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572541.656879:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572541.656899:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1621e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.656926:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1621d9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6dc expref: 99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.656944:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1621e0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6dc expref: 98 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.656953:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:2.0:1788572541.656956:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572541.656960:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:2.0:1788572541.657053:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35902:x1875453360331264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 411us (730us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.657061:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4022 00000100:00100000:1.0:1788572541.657488:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.657684:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331648 00000100:00100000:0.0:1788572541.657690:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331648 00000100:00100000:0.0:1788572541.657739:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4023 00000100:00100000:0.0:1788572541.657749:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35901:x1875453360331648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.657761:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.657775:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1621e7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6e3 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.657822:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1621ee lrc: 3/1,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.657833:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1621ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.657843:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1621ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.657869:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.657882:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1621ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.657903:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1621e7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6e3 expref: 100 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.657922:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1621ee lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6e3 expref: 99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.657931:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572541.657933:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572541.657936:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572541.657968:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35901:x1875453360331648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 221us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.657974:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4023 00000100:00100000:3.0:1788572541.659429:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331520 00000100:00100000:3.0:1788572541.659443:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331520 00000100:00100000:3.0:1788572541.659543:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 878 00000100:00100000:3.0:1788572541.659553:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35900:x1875453360331520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.659983:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35900:x1875453360331520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 431us (3542us total) trans 4294970319 rc 0/0 00000100:00100000:3.0:1788572541.659990:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 878 00000100:00100000:1.0:1788572541.687713:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.687762:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.687843:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.687878:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.687936:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331776 00000100:00100000:2.0:1788572541.687947:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331776 00002000:00100000:2.0:1788572541.688037:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x43:0x0] (0->4095) req@ffff962801f94a80 x1875453360331776/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.688049:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f94a80 with x1875453360331776 ext(0->4095) 00002000:00010000:2.0:1788572541.688055:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff962801f94a80 00000100:00100000:2.0:1788572541.688085:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 238 00000100:00100000:2.0:1788572541.688096:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+34:2349:x1875453360331776:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572541.688330:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+34:2349:x1875453360331776:12345-192.168.202.40@tcp:3:diff.0 Request processed in 235us (626us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572541.688355:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 238 00002000:00100000:2.0:1788572541.688361:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x43:0x0] (0->4095) req@ffff962801f94a80 x1875453360331776/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.688370:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f94a80 with x1875453360331776 ext(0->4095) 00002000:00010000:2.0:1788572541.688373:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff962801f94a80 00000100:00100000:0.0:1788572541.689354:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360332160 00000100:00100000:0.0:1788572541.689364:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360332160 00000100:00100000:0.0:1788572541.689426:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4024 00000100:00100000:0.0:1788572541.689436:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35900:x1875453360332160:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572541.689564:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35900:x1875453360332160:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 130us (1721us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.689573:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4024 00000100:00100000:3.0:1788572541.692328:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360331904 00000100:00100000:3.0:1788572541.692359:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360331904 00000100:00100000:3.0:1788572541.692456:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360332288 00000100:00100000:3.0:1788572541.692470:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360332288 00000100:00100000:0.0:1788572541.694095:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1044 00000100:00100000:0.0:1788572541.694118:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360331904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.694139:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.694161:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6ea expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.694189:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x46:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737d, blocks: 0x1 00010000:00010000:0.0:1788572541.694210:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=889s v=(0 0) 00010000:00010000:0.0:1788572541.694215:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.694226:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.694252:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.694266:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572541.694772:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360331904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 659us (7004us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.694781:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1044 00000100:00100000:0.0:1788572541.694812:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1045 00000100:00100000:0.0:1788572541.694823:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360332288:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.694834:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.694846:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6f1 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.694864:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x47:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737d, blocks: 0x1 00010000:00010000:0.0:1788572541.694882:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=889s v=(0 0) 00010000:00010000:0.0:1788572541.694886:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.694892:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.694906:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.694914:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d400, rc 0) 00000100:00100000:0.0:1788572541.694945:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360332288:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 125us (7066us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.694952:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1045 00000100:00100000:2.0:1788572541.695143:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.695262:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360332544 00000100:00100000:1.0:1788572541.695271:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360332544 00000100:00100000:2.0:1788572541.695421:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 879 00000100:00100000:2.0:1788572541.695432:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360332544:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572541.695872:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360332544:12345-192.168.202.40@tcp:35:diff.0 Request processed in 441us (729us total) trans 4294970320 rc 0/0 00000100:00100000:2.0:1788572541.695881:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 879 00000100:00100000:0.0:1788572541.696855:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.696978:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360332800 00000100:00100000:1.0:1788572541.696986:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360332800 00000100:00100000:1.0:1788572541.697049:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4025 00000100:00100000:1.0:1788572541.697059:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35902:x1875453360332800:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.697157:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35902:x1875453360332800:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 100us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.697166:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4025 00000100:00100000:2.0:1788572541.701045:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.701134:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333056 00000100:00100000:1.0:1788572541.701148:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333056 00000100:00100000:1.0:1788572541.701216:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4026 00000100:00100000:1.0:1788572541.701228:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35901:x1875453360333056:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.701357:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35901:x1875453360333056:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 131us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.701365:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4026 00000100:00100000:1.0:1788572541.705094:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.705379:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333184 00000100:00100000:0.0:1788572541.705390:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333184 00000100:00100000:0.0:1788572541.705466:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4027 00000100:00100000:0.0:1788572541.705477:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35900:x1875453360333184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572541.705499:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.705521:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094da00/0x85ace67feb162203 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6f8 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.705599:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c800/0x85ace67feb16220a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.705615:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c800/0x85ace67feb16220a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.705626:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c800/0x85ace67feb16220a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.705917:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572541.705943:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c800/0x85ace67feb16220a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.705976:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094da00/0x85ace67feb162203 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f8 expref: 102 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.705999:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c800/0x85ace67feb16220a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f8 expref: 101 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.706466:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:0.0:1788572541.706470:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572541.706474:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c800, rc 0) 00000100:00100000:0.0:1788572541.706549:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35900:x1875453360333184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1074us (1466us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.706557:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4027 00000100:00100000:0.0:1788572541.718134:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.718166:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.718191:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.718231:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.718325:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333440 00000100:00100000:2.0:1788572541.718345:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333440 00000100:00100000:2.0:1788572541.718393:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333568 00000100:00100000:2.0:1788572541.718397:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333568 00000100:00100000:2.0:1788572541.718415:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333696 00000100:00100000:2.0:1788572541.718419:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333696 00000100:00100000:2.0:1788572541.718439:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1046 00000100:00100000:2.0:1788572541.718447:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35901:x1875453360333440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.718477:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.718498:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef706 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718533:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718543:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.718547:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.718553:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718590:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718607:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.718624:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382347776:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572541.718859:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff3fee40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572541.718883:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1047 00000100:00100000:2.0:1788572541.718891:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35902:x1875453360333568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.718900:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.718911:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef70d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718929:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718937:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.718940:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572541.718944:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 15 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718965:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.718975:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.718986:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382347904:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572541.719039:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0cebe40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.719112:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1048 00000100:00100000:3.0:1788572541.719122:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360333696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.719131:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.719143:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef714 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572541.719169:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x47:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737d, blocks: 0x1 00010000:00010000:3.0:1788572541.719181:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=889s v=(0 0) 00010000:00010000:3.0:1788572541.719184:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572541.719192:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.719209:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.719221:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051600, rc 0) 00000100:00100000:3.0:1788572541.719251:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360333696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 131us (1019us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.719258:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1048 00000100:00100000:1.0:1788572541.719495:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333312 00000100:00100000:1.0:1788572541.719501:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333312 00000100:00100000:1.0:1788572541.719543:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4028 00000100:00100000:1.0:1788572541.719552:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360333312:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572541.719565:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.719579:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162226 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef6ff expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.719860:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.719877:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.719888:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.719904:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.719911:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.719918:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720257:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb16223b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720266:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb16223b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720274:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb16223b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572541.720307:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbf:0x0]: returning size 3 00000004:00200000:1.0:1788572541.720311:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572541.720635:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb16223b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720718:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb16223b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720732:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720739:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720745:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720751:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.720755:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720764:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720776:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:1.0:1788572541.720779:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.720784:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720791:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16222d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720805:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720812:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720817:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720823:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bac400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.720826:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720832:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720839:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x74f887a3 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572541.720841:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.720843:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720849:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bac400/0x85ace67feb162234 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720884:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162226 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ff expref: 169 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720897:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246d800/0x85ace67feb16223b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ff expref: 168 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.720905:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:1.0:1788572541.720908:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572541.720910:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246d800, rc 0) 00000100:00100000:2.0:1788572541.721951:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:35867:x1875453360333312:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2392us (3822us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.721961:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4028 00000100:00100000:0.0:1788572541.724883:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.724987:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382347776:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.725047:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360333824 00010000:00010000:2.0:1788572541.725052:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.725055:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360333824 00010000:00010000:2.0:1788572541.725074:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53600, rc 0) 00010000:00010000:1.0:1788572541.725087:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 15 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.725129:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35901:x1875453360333440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6682us (6961us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.725135:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 955 00000100:00100000:2.0:1788572541.725136:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1046 00000100:00100000:1.0:1788572541.725145:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360333824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.725163:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.725188:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.725202:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.725230:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b219a60/0x85ace67feb1621fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f1 expref: 14 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.725250:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.725255:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.725264:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.725275:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.725312:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.725329:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382348032:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572541.725410:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57a0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572541.729941:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.730112:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334080 00000100:00100000:1.0:1788572541.730122:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334080 00000100:00100000:1.0:1788572541.730174:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4029 00000100:00100000:1.0:1788572541.730185:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35900:x1875453360334080:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.730312:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35900:x1875453360334080:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 128us (393us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.730320:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4029 00000100:00100000:2.0:1788572541.730474:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382347904:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572541.730509:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 15 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.730534:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:2.0:1788572541.730563:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35902:x1875453360333568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 11673us (12370us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.730571:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1047 00000100:00100000:0.0:1788572541.734681:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.734727:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.734864:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334336 00000100:00100000:3.0:1788572541.734875:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334336 00000100:00100000:3.0:1788572541.734949:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1049 00000100:00100000:3.0:1788572541.734959:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2350:x1875453360334336:12345-192.168.202.40@tcp:101:diff.0 00000100:00100000:1.0:1788572541.734971:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334208 00000100:00100000:1.0:1788572541.734975:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334208 00010000:00010000:3.0:1788572541.734977:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.734997:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.734997:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e12d23a3/0x85ace67feb162242 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef71b expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572541.735056:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x40:0x0] initial LVB size: 3, mtime: 0x6a9b7363, atime: 0x6a9b7363, ctime: 0x6a9b736f, blocks: 0xb 00000100:00100000:1.0:1788572541.735057:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 956 00000100:00100000:1.0:1788572541.735064:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360334208:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.735074:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572541.735077:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=889s v=(0 0) 00010000:00010000:3.0:1788572541.735080:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.735089:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.735089:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e12d23a3/0x85ace67feb162242 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef71b expref: 44 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.735103:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.735107:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e12d23a3/0x85ace67feb162242 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef71b expref: 44 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.735121:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb1621f5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ea expref: 15 pid: 6681 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.735125:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051c00, rc 0) 00010000:00010000:1.0:1788572541.735144:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.735147:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.735154:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.735162:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.735174:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+44:2350:x1875453360334336:12345-192.168.202.40@tcp:101:diff.0 Request processed in 218us (447us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.735181:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1049 00010000:00010000:1.0:1788572541.735194:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.735222:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382348160:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572541.735277:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.738147:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.738238:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.738359:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334592 00000100:00100000:2.0:1788572541.738368:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334592 00002000:00100000:2.0:1788572541.738419:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x47:0x0] (0->2) req@ffff9627ca8c9880 x1875453360334592/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.738431:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8c9880 with x1875453360334592 ext(0->2) 00002000:00010000:2.0:1788572541.738437:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ca8c9880 00000100:00100000:2.0:1788572541.738479:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 239 00000100:00100000:2.0:1788572541.738490:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360334592:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.738955:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360334592:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 466us (715us total) trans 4294967570 rc 0/0 00000100:00100000:2.0:1788572541.738963:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 239 00002000:00100000:2.0:1788572541.738970:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x47:0x0] (0->2) req@ffff9627ca8c9880 x1875453360334592/t4294967570(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/448 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.738979:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8c9880 with x1875453360334592 ext(0->2) 00002000:00010000:2.0:1788572541.738982:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ca8c9880 00000100:00100000:0.0:1788572541.739526:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382348032:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572541.739550:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572541.739576:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360333824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 14432us (14704us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.739585:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 955 00000100:00100000:1.0:1788572541.739877:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334464 00000100:00100000:1.0:1788572541.739884:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334464 00000100:00100000:1.0:1788572541.739917:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1050 00000100:00100000:1.0:1788572541.739924:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35900:x1875453360334464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.739935:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.739953:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef722 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.739980:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.739994:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.739998:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.740038:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.740072:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.740085:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 16 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.740100:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382348288:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.740170:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628008d8900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.741312:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382348160:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572541.741331:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.741366:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360334208:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6302us (6697us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.741375:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 956 00000100:00100000:3.0:1788572541.743536:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.743770:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334848 00000100:00100000:0.0:1788572541.743781:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334848 00000100:00100000:0.0:1788572541.743848:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4030 00000100:00100000:0.0:1788572541.743860:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:35867:x1875453360334848:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572541.743882:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.743904:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162250 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef729 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.744079:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162257 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.744095:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162257 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.744106:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162257 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.744119:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbf:0x0]: returning size 3 00010000:00010000:0.0:1788572541.744253:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162250 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef729 expref: 171 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.744278:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162257 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef729 expref: 170 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.744286:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:0.0:1788572541.744289:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.744294:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:0.0:1788572541.744365:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360334848:12345-192.168.202.40@tcp:101:diff.0 Request processed in 507us (841us total) trans 4294970321 rc 0/0 00000100:00100000:0.0:1788572541.744373:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4030 00000100:00100000:0.0:1788572541.760226:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.760278:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.760342:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.760356:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360334976 00000100:00100000:2.0:1788572541.760366:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360334976 00002000:00100000:2.0:1788572541.760425:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x46:0x0] (0->2) req@ffff9627c9a58a80 x1875453360334976/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.760437:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a58a80 with x1875453360334976 ext(0->2) 00002000:00010000:2.0:1788572541.760444:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a58a80 00000100:00100000:2.0:1788572541.760516:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 240 00000100:00100000:2.0:1788572541.760526:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2347:x1875453360334976:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.761473:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2347:x1875453360334976:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 945us (1254us total) trans 4294967568 rc 0/0 00000100:00100000:2.0:1788572541.761483:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 240 00002000:00100000:2.0:1788572541.761492:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x46:0x0] (0->2) req@ffff9627c9a58a80 x1875453360334976/t4294967568(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/448 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1788572541.761497:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360335232 00000100:00100000:1.0:1788572541.761502:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360335232 00002000:00100000:2.0:1788572541.761508:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a58a80 with x1875453360334976 ext(0->2) 00002000:00010000:2.0:1788572541.761512:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a58a80 00010000:00010000:1.0:1788572541.761528:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.761556:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 957 00000100:00100000:1.0:1788572541.761564:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a59180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360335232:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.761576:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.761591:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.761606:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.761631:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.761635:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.761657:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 18 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.761667:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 18 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.761703:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 18 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.761717:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382348416:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572541.761770:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00010000:00010000:1.0:1788572541.761807:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb16221f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef714 expref: 18 pid: 20911 timeout: 989 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.761822:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382348288:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.761847:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 17 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.761858:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246c000, rc 0) 00000100:00100000:1.0:1788572541.761879:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35900:x1875453360334464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 21956us (23743us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.761886:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1050 00000100:00100000:3.0:1788572541.762567:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360335104 00000100:00100000:3.0:1788572541.762575:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360335104 00000100:00100000:3.0:1788572541.762623:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4031 00000100:00100000:3.0:1788572541.762632:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35901:x1875453360335104:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.771423:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.774051:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360335360 00000100:00100000:2.0:1788572541.774068:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360335360 00002000:00100000:2.0:1788572541.774124:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x40:0x0] (0->4095) req@ffff9627dfdbf480 x1875453360335360/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.774137:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbf480 with x1875453360335360 ext(0->4095) 00002000:00010000:2.0:1788572541.774142:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbf480 00000100:00100000:2.0:1788572541.774169:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 241 00000100:00100000:2.0:1788572541.774177:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+44:2348:x1875453360335360:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572541.774380:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+44:2348:x1875453360335360:12345-192.168.202.40@tcp:3:diff.0 Request processed in 202us (2966us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572541.774388:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 241 00002000:00100000:2.0:1788572541.774395:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x40:0x0] (0->4095) req@ffff9627dfdbf480 x1875453360335360/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572541.774403:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbf480 with x1875453360335360 ext(0->4095) 00002000:00010000:2.0:1788572541.774407:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbf480 00000100:00100000:0.0:1788572541.779407:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.779436:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.781183:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382348416:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572541.781210:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572541.781229:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.781266:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a59180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360335232:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 19702us (20922us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.781278:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 957 00000100:00100000:1.0:1788572541.781309:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360335616 00000100:00100000:1.0:1788572541.781315:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360335616 00000100:00100000:1.0:1788572541.781356:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 301 00000100:00100000:1.0:1788572541.781365:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e53850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:2350:x1875453360335616:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.781380:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.781405:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162257 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef729 expref: 170 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.781423:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffff9627d0ee2500) 00010000:00010000:1.0:1788572541.781428:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.781470:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e53850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:2350:x1875453360335616:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 95us (2022us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.781481:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 301 00000100:00100000:1.0:1788572541.781527:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360335488 00000100:00100000:1.0:1788572541.781532:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360335488 00000100:00100000:1.0:1788572541.781561:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4032 00000100:00100000:1.0:1788572541.781568:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35902:x1875453360335488:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.782085:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360335744 00000100:00100000:3.0:1788572541.782092:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360335744 00002000:00100000:3.0:1788572541.782124:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x47:0x0] (0->2) req@ffff9627c9a59880 x1875453360335744/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572541.782135:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a59880 with x1875453360335744 ext(0->2) 00002000:00010000:3.0:1788572541.782140:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a59880 00000100:00100000:3.0:1788572541.782167:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 242 00000100:00100000:3.0:1788572541.782175:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360335744:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.787114:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.787289:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35902:x1875453360335488:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 5718us (7889us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.787300:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4032 00000100:00100000:0.0:1788572541.787492:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336000 00000100:00100000:0.0:1788572541.787501:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336000 00000100:00100000:0.0:1788572541.787546:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 880 00000100:00100000:0.0:1788572541.787557:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360336000:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572541.789147:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360336000:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1581us (2033us total) trans 4294970322 rc 0/0 00000100:00100000:0.0:1788572541.789159:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 880 00000100:00100000:3.0:1788572541.792541:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360335744:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 10355us (11300us total) trans 4294967569 rc 0/0 00000100:00100000:3.0:1788572541.792555:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 242 00002000:00100000:3.0:1788572541.792569:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x47:0x0] (0->2) req@ffff9627c9a59880 x1875453360335744/t4294967569(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:202/0 lens 496/448 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572541.792579:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a59880 with x1875453360335744 ext(0->2) 00002000:00010000:3.0:1788572541.792586:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a59880 00000100:00100000:2.0:1788572541.802085:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.802132:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.802207:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.802291:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336128 00000100:00100000:3.0:1788572541.802303:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336128 00000100:00100000:3.0:1788572541.802359:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336384 00000100:00100000:3.0:1788572541.802363:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336384 00000100:00100000:3.0:1788572541.802384:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4033 00000100:00100000:3.0:1788572541.802394:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360336128:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572541.802420:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.802466:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb162265 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef730 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802546:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802561:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802573:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802588:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802596:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802603:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802915:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb16227a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802924:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb16227a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.802932:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb16227a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.802976:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbe:0x0]: returning size 3 00000004:00200000:3.0:1788572541.802981:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.803035:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb16227a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803048:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb16227a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803057:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803062:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803068:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803073:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.803077:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803084:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803096:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.803099:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.803104:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803109:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb16226c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803120:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803126:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803131:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803136:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.803138:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803143:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803149:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xbd5240a3 (ffff9627ffc51300) 00010000:00010000:3.0:1788572541.803152:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.803154:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803160:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb162273 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803196:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb162265 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef730 expref: 172 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803210:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90600/0x85ace67feb16227a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef730 expref: 171 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.803218:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627ffc51400) 00010000:00010000:3.0:1788572541.803220:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.803222:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90600, rc 0) 00000100:00100000:3.0:1788572541.803285:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360336128:12345-192.168.202.40@tcp:101:diff.0 Request processed in 892us (1210us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.803293:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4033 00000100:00100000:3.0:1788572541.803316:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4034 00000100:00100000:3.0:1788572541.803324:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35900:x1875453360336384:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.803380:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35900:x1875453360336384:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 56us (1172us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.803386:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4034 00000100:00100000:1.0:1788572541.804555:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336256 00000100:00100000:1.0:1788572541.804567:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336256 00000100:00100000:1.0:1788572541.804607:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1051 00000100:00100000:1.0:1788572541.804614:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360336256:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572541.814120:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.814153:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.815323:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336512 00000100:00100000:1.0:1788572541.815338:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336512 00000100:00100000:1.0:1788572541.815394:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336640 00000100:00100000:1.0:1788572541.815398:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336640 00000100:00100000:1.0:1788572541.815424:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1052 00000100:00100000:1.0:1788572541.815433:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+34:2347:x1875453360336512:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572541.815454:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572541.815485:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000049f900c8/0x85ace67feb162281 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef737 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572541.815514:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x44:0x0] initial LVB size: 3, mtime: 0x6a9b7363, atime: 0x6a9b7363, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:1.0:1788572541.815534:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=889s v=(0 0) 00010000:00010000:1.0:1788572541.815539:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572541.815548:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000049f900c8/0x85ace67feb162281 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef737 expref: 35 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.815566:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000049f900c8/0x85ace67feb162281 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef737 expref: 35 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.815580:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bac400, rc 0) 00000100:00100000:1.0:1788572541.815643:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+35:2347:x1875453360336512:12345-192.168.202.40@tcp:101:diff.0 Request processed in 211us (1535us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.819165:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1053 00000100:00100000:3.0:1788572541.819186:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360336640:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572541.819309:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1052 00000100:00100000:2.0:1788572541.831469:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.831612:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360336896 00000100:00100000:0.0:1788572541.831621:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360336896 00000100:00100000:0.0:1788572541.831688:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4035 00000100:00100000:0.0:1788572541.831698:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360336896:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572541.831721:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.831748:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162288 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef73e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.831863:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16228f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.831876:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16228f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.831889:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16228f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572541.831900:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbe:0x0]: returning size 3 00010000:00010000:0.0:1788572541.832216:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162288 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef73e expref: 174 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.832243:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16228f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef73e expref: 173 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.832253:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627ffc51400) 00010000:00010000:0.0:1788572541.832256:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572541.832260:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:0.0:1788572541.832399:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360336896:12345-192.168.202.40@tcp:101:diff.0 Request processed in 701us (949us total) trans 4294970323 rc 0/0 00000100:00100000:0.0:1788572541.832407:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4035 00000100:00100000:3.0:1788572541.855027:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.855067:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.855538:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360337152 00000100:00100000:1.0:1788572541.855549:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360337152 00000100:00100000:1.0:1788572541.855619:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 302 00000100:00100000:1.0:1788572541.855630:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:2347:x1875453360337152:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.855649:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.855674:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16228f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef73e expref: 173 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.855697:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffff9627ffc51400) 00010000:00010000:1.0:1788572541.855704:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.855772:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:2347:x1875453360337152:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 144us (704us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.855780:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 302 00000100:00100000:3.0:1788572541.856957:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35901:x1875453360335104:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 94319us (96671us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572541.856968:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4031 00000100:00100000:3.0:1788572541.859039:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360337024 00000100:00100000:3.0:1788572541.859055:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360337024 00002000:00100000:3.0:1788572541.859108:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x44:0x0] (0->4095) req@ffff9627ca8c8e00 x1875453360337024/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572541.859120:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8c8e00 with x1875453360337024 ext(0->4095) 00002000:00010000:3.0:1788572541.859127:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ca8c8e00 00000100:00100000:3.0:1788572541.859165:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 243 00000100:00100000:3.0:1788572541.859175:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+35:2349:x1875453360337024:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572541.859380:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+35:2349:x1875453360337024:12345-192.168.202.40@tcp:3:diff.0 Request processed in 206us (4378us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572541.859389:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 243 00002000:00100000:3.0:1788572541.859396:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x44:0x0] (0->4095) req@ffff9627ca8c8e00 x1875453360337024/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572541.859405:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8c8e00 with x1875453360337024 ext(0->4095) 00002000:00010000:3.0:1788572541.859408:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ca8c8e00 00000001:00080000:2.0:1788572541.859838:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970321 is committed 00000001:00080000:2.0:1788572541.859852:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970312 is committed 00000004:00080000:2.0:1788572541.859857:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 35 committed 00000020:00080000:2.0:1788572541.859892:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c97cb0c0 refcount 2 committed 0 result 0 batchid 105 00000020:00080000:2.0:1788572541.859896:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceab00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c97cb000 00000020:00080000:2.0:1788572541.859900:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceaf00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff091800 00000020:00080000:0.0:1788572541.859950:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 99 00000100:00100000:3.0:1788572541.860291:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382348672:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572541.860353:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572541.860414:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788572541.860421:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382348544:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572541.860446:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572541.860500:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1788572541.860509:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382348800:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572541.860536:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572541.860566:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382348672 00000100:00100000:3.0:1788572541.860572:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382348672 00000100:00100000:3.0:1788572541.860603:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382348544 00000100:00100000:3.0:1788572541.860607:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382348544 00000100:00100000:3.0:1788572541.860619:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1054 00000100:00100000:3.0:1788572541.860627:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382348672:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572541.860875:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3305:x1875453382348672:12345-0@lo:2:osp-syn-1-0.0 Request processed in 248us (520us total) trans 4294967570 rc 0/0 00000100:00100000:3.0:1788572541.860883:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1054 00000100:00100000:3.0:1788572541.860897:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382348800 00000100:00100000:3.0:1788572541.860900:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382348800 00000100:00100000:3.0:1788572541.860913:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1055 00000100:00100000:3.0:1788572541.860918:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3305:x1875453382348544:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572541.861407:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1056 00000100:00100000:0.0:1788572541.861418:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382348800:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1788572541.861532:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572541.861536:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb mtime from disk: 0 -> 1788572541 00002000:00010000:0.0:1788572541.861539:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1788572541.861541:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb ctime from disk: 1788572541 -> 1788572541 00002000:00010000:0.0:1788572541.861544:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572541.861584:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382348800:12345-0@lo:2:osp-syn-1-0.0 Request processed in 168us (1046us total) trans 4294967571 rc 0/0 00000100:00100000:0.0:1788572541.861591:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1056 00002000:00010000:3.0:1788572541.861722:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572541.861725:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb mtime from disk: 1788572515 -> 1788572515 00002000:00010000:3.0:1788572541.861727:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb atime from disk: 1788572515 -> 1788572515 00002000:00010000:3.0:1788572541.861729:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb ctime from disk: 1788572527 -> 1788572527 00002000:00010000:3.0:1788572541.861731:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x43:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572541.861758:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382348544:12345-0@lo:2:osp-syn-1-0.0 Request processed in 839us (1310us total) trans 4294967572 rc 0/0 00000100:00100000:3.0:1788572541.861764:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1055 00000004:00080000:3.0:1788572541.861809:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627d131d180/2, rc 0, transno 4294967572 00000100:00100000:3.0:1788572541.861829:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382348544:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572541.861841:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfae7100/2, rc 0, transno 4294967570 00000100:00100000:3.0:1788572541.861845:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382348672:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572541.861865:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfae4a80/2, rc 0, transno 4294967571 00000100:00100000:3.0:1788572541.861869:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382348800:0@lo:2:osp-syn-1-0.0 00000020:00080000:2.0:1788572541.861998:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ff3d4840 refcount 2 committed 0 result 0 batchid 103 00000020:00080000:2.0:1788572541.862015:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae800 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2f56840 00000020:00080000:2.0:1788572541.862019:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c10a9400 00000001:00080000:2.0:1788572541.862047:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970315 is committed 00000004:00080000:2.0:1788572541.862056:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 40 committed 00000020:00080000:2.0:1788572541.862074:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c978d3c0 refcount 2 committed 0 result 0 batchid 104 00000020:00080000:2.0:1788572541.862078:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ff598800 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c978d180 00000020:00080000:2.0:1788572541.862080:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ff598c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f59a1c00 00000020:00080000:2.0:1788572541.862094:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 102 batchid updated 00000001:00080000:2.0:1788572541.862100:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970319 is committed 00000100:00100000:2.0:1788572541.862497:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382348928:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572541.862541:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572541.862587:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382349184:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572541.862610:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572541.862629:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572541.862635:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382349056:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572541.862654:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572541.862676:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382348928 00000100:00100000:2.0:1788572541.862680:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382348928 00000100:00100000:2.0:1788572541.862706:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382349184 00000100:00100000:2.0:1788572541.862709:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382349184 00000100:00100000:2.0:1788572541.862719:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382349056 00000100:00100000:2.0:1788572541.862722:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382349056 00000100:00100000:2.0:1788572541.862733:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1057 00000100:00100000:2.0:1788572541.862740:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382348928:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572541.862853:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572541.862857:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb mtime from disk: 1788572516 -> 1788572516 00002000:00010000:2.0:1788572541.862859:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb atime from disk: 1788572516 -> 1788572516 00002000:00010000:2.0:1788572541.862861:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb ctime from disk: 1788572527 -> 1788572527 00002000:00010000:2.0:1788572541.862863:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x45:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572541.862899:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382348928:12345-0@lo:2:osp-syn-0-0.0 Request processed in 158us (358us total) trans 4294967571 rc 0/0 00000100:00100000:2.0:1788572541.862906:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1057 00000100:00100000:2.0:1788572541.862923:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1058 00000100:00100000:0.0:1788572541.863046:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1059 00000100:00100000:0.0:1788572541.863057:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382349056:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1788572541.863137:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:0.0:1788572541.863140:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb mtime from disk: 1788572515 -> 1788572515 00002000:00010000:0.0:1788572541.863143:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb atime from disk: 1788572515 -> 1788572515 00002000:00010000:0.0:1788572541.863145:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb ctime from disk: 1788572527 -> 1788572527 00002000:00010000:0.0:1788572541.863148:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x46:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:0.0:1788572541.863182:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382349056:12345-0@lo:2:osp-syn-0-0.0 Request processed in 129us (527us total) trans 4294967572 rc 0/0 00000100:00100000:0.0:1788572541.863189:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1059 00000020:00080000:3.0:1788572541.863344:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 105 committed batchid 102 00000020:00080000:3.0:1788572541.863519:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 102 00000100:00100000:2.0:1788572541.863677:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382349184:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572541.863757:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572541.863761:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb mtime from disk: 0 -> 1788572541 00002000:00010000:2.0:1788572541.863763:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1788572541.863765:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb ctime from disk: 1788572541 -> 1788572541 00002000:00010000:2.0:1788572541.863768:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1788572541.863803:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382349184:12345-0@lo:2:osp-syn-0-0.0 Request processed in 132us (1191us total) trans 4294967573 rc 0/0 00000100:00100000:2.0:1788572541.863810:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1058 00000004:00080000:2.0:1788572541.863855:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9626c12eea00/2, rc 0, transno 4294967571 00000100:00100000:2.0:1788572541.863861:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382348928:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572541.863881:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9626c1bcc000/2, rc 0, transno 4294967572 00000100:00100000:2.0:1788572541.863886:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382349056:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572541.863893:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff962801f92300/2, rc 0, transno 4294967573 00000100:00100000:2.0:1788572541.863896:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382349184:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572541.864535:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.864611:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360337408 00000100:00100000:3.0:1788572541.864617:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360337408 00000100:00100000:3.0:1788572541.864665:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 881 00000100:00100000:3.0:1788572541.864674:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+164:35867:x1875453360337408:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572541.865192:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:35867:x1875453360337408:12345-192.168.202.40@tcp:35:diff.0 Request processed in 518us (659us total) trans 4294970324 rc 0/0 00000100:00100000:3.0:1788572541.865201:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 881 00000100:00100000:3.0:1788572541.872224:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.872317:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.872378:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360337536 00000100:00100000:0.0:1788572541.872388:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360337536 00000100:00100000:0.0:1788572541.872441:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1060 00000100:00100000:0.0:1788572541.872452:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360337536:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572541.873101:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360337664 00000100:00100000:2.0:1788572541.873109:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360337664 00000100:00100000:2.0:1788572541.873152:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4036 00000100:00100000:2.0:1788572541.873161:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:35867:x1875453360337664:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572541.873179:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.873202:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb16229d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef745 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873282:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873299:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873311:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873327:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873334:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873341:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873632:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1622b2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873645:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1622b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873653:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1622b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572541.873686:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbd:0x0]: returning size 3 00000004:00200000:2.0:1788572541.873690:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572541.873711:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1622b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873723:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1622b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873731:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873737:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873744:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873750:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.873754:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873762:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873774:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572541.873778:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.873782:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873789:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1622a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873802:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873807:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873813:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873818:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572541.873821:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873826:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873834:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x5abf9a3 (ffff9627e7a95900) 00010000:00010000:2.0:1788572541.873836:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572541.873839:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873844:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1622ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873875:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb16229d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef745 expref: 167 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873893:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53000/0x85ace67feb1622b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef745 expref: 166 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.873900:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627e7a95d00) 00010000:00010000:2.0:1788572541.873903:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572541.873905:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53000, rc 0) 00000100:00100000:2.0:1788572541.873974:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360337664:12345-192.168.202.40@tcp:101:diff.0 Request processed in 814us (1657us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.873982:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4036 00000001:00080000:2.0:1788572541.878777:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967564 is committed 00000001:00080000:2.0:1788572541.878805:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967568 is committed 00000001:00080000:2.0:1788572541.878831:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967569 is committed 00000001:00080000:2.0:1788572541.878840:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967565 is committed 00000100:00100000:2.0:1788572541.878963:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360336640:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 59779us (64804us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.878974:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1053 00000100:00100000:1.0:1788572541.879764:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360336256:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 75146us (77627us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.879778:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1051 00000100:00100000:1.0:1788572541.886109:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.886228:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360337920 00000100:00100000:3.0:1788572541.886238:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360337920 00000100:00100000:3.0:1788572541.886295:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4037 00000100:00100000:3.0:1788572541.886306:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35902:x1875453360337920:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.886367:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572541.886394:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1621e0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6dc expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.886414:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:3.0:1788572541.886419:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572541.886452:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.886476:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.886492:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572541.886652:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bab44556 reply state 00000000c12902a1 transno 4294970325 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.886662:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572541.886759:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35902:x1875453360337920:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 454us (658us total) trans 4294970325 rc 0/0 00000100:00100000:3.0:1788572541.886767:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4037 00000100:00100000:1.0:1788572541.887537:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.887691:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360338176 00000100:00100000:3.0:1788572541.887697:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360338176 00000100:00100000:3.0:1788572541.887731:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4038 00000100:00100000:3.0:1788572541.887738:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35900:x1875453360338176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572541.887770:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572541.887787:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c800/0x85ace67feb16220a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6f8 expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.887805:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:3.0:1788572541.887808:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572541.887827:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.887838:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.887858:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572541.887970:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008367e449 reply state 00000000e38b787e transno 4294970326 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572541.887978:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572541.888059:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35900:x1875453360338176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 320us (525us total) trans 4294970326 rc 0/0 00000100:00100000:3.0:1788572541.888066:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4038 00000100:00100000:1.0:1788572541.891476:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572541.891528:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.891713:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360338432 00000100:00100000:3.0:1788572541.891724:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360338432 00000100:00100000:3.0:1788572541.891776:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 271 00000100:00100000:3.0:1788572541.891787:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:2348:x1875453360338432:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572541.891828:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.891855:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1622c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef753 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.891921:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.891935:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.891947:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.891963:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.891969:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.891976:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.892328:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2800/0x85ace67feb1622dc lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.892919:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360338304 00000100:00100000:0.0:1788572541.892937:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360338304 00000100:00100000:0.0:1788572541.892984:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1061 00000100:00100000:0.0:1788572541.892992:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+44:2347:x1875453360338304:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572541.893022:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.893037:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004b219a60/0x85ace67feb1622e3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef74c expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.893059:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x45:0x0] initial LVB size: 3, mtime: 0x6a9b7363, atime: 0x6a9b7363, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:0.0:1788572541.893074:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=889s v=(0 0) 00010000:00010000:0.0:1788572541.893079:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x45:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.893089:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004b219a60/0x85ace67feb1622e3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef74c expref: 45 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.893104:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004b219a60/0x85ace67feb1622e3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x45:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef74c expref: 45 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.893124:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d400, rc 0) 00000100:00100000:0.0:1788572541.893177:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+45:2347:x1875453360338304:12345-192.168.202.40@tcp:101:diff.0 Request processed in 186us (1728us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.893185:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1061 00010000:00010000:3.0:1788572541.893494:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2800/0x85ace67feb1622dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893503:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2800/0x85ace67feb1622dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.893533:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbc:0x0]: returning size 3 00000004:00200000:3.0:1788572541.893537:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.893557:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2800/0x85ace67feb1622dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893569:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2800/0x85ace67feb1622dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893579:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893586:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893593:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893599:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.893604:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893613:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.893625:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.893628:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.893632:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897621:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb1622ce lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897667:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897676:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897684:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897689:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.897695:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897704:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897720:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x4e05b2a3 (ffff9627ffc51300) 00010000:00010000:3.0:1788572541.897724:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.897729:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897739:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb1622d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897791:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb1622c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef753 expref: 168 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897807:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2800/0x85ace67feb1622dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef753 expref: 167 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897814:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627ffc51600) 00010000:00010000:3.0:1788572541.897817:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.897819:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea2800, rc 0) 00010000:00010000:3.0:1788572541.897832:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.897852:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1622ea lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef75a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897903:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897916:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897926:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897941:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897948:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.897954:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898283:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2000/0x85ace67feb1622ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898293:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2000/0x85ace67feb1622ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898300:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2000/0x85ace67feb1622ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.898319:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbb:0x0]: returning size 3 00000004:00200000:3.0:1788572541.898323:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.898342:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2000/0x85ace67feb1622ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898354:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2000/0x85ace67feb1622ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898361:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898366:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898372:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898376:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.898379:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898384:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898392:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.898395:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.898398:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898403:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb1622f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898411:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898416:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898422:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898427:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.898429:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898434:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898440:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x965f6ba3 (ffff9627ffc51200) 00010000:00010000:3.0:1788572541.898441:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.898444:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898452:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1622f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898491:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1622ea lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef75a expref: 169 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898501:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2000/0x85ace67feb1622ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef75a expref: 168 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898507:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627ffc51f00) 00010000:00010000:3.0:1788572541.898509:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.898511:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea2000, rc 0) 00010000:00010000:3.0:1788572541.898516:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.898537:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162306 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef761 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898564:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898572:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898579:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898589:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898596:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898603:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898764:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd000/0x85ace67feb16231b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898773:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd000/0x85ace67feb16231b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898780:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd000/0x85ace67feb16231b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572541.898794:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xba:0x0]: returning size 3 00000004:00200000:3.0:1788572541.898797:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572541.898807:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd000/0x85ace67feb16231b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898815:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd000/0x85ace67feb16231b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898821:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898827:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898832:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898837:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.898840:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898846:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898854:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.898856:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.898859:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898864:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16230d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898874:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898879:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898884:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898889:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.898891:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898896:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898903:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xdeb924a3 (ffff9627ffc51700) 00010000:00010000:3.0:1788572541.898905:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.898908:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898918:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162314 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898938:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162306 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef761 expref: 170 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898949:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd000/0x85ace67feb16231b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef761 expref: 169 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898956:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627ffc51e00) 00010000:00010000:3.0:1788572541.898958:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572541.898960:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd000, rc 0) 00010000:00010000:3.0:1788572541.898965:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572541.898972:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162322 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef768 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.898995:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899028:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899036:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899047:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899055:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899063:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899141:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899148:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899153:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899158:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.899161:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899167:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899176:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.899178:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.899181:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899187:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162329 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899195:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899201:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899207:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899213:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572541.899216:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899222:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899229:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xc8545ca3 (ffff9627ffc51000) 00010000:00010000:3.0:1788572541.899231:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572541.899233:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899238:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162330 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899260:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162322 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef768 expref: 170 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899270:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572541.899273:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572541.899276:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162322 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef768 expref: 170 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572541.899286:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc -2) 00000100:00100000:3.0:1788572541.899365:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:2348:x1875453360338432:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 7579us (7835us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572541.899378:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 271 00000100:00100000:3.0:1788572541.912543:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.912569:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572541.912593:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360338560 00000100:00100000:3.0:1788572541.912602:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360338560 00000100:00100000:3.0:1788572541.912661:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1062 00000100:00100000:3.0:1788572541.912672:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360338560:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572541.912866:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572541.912870:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb mtime from disk: 1788572541 -> 1788572520 00002000:00010000:3.0:1788572541.912872:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:3.0:1788572541.912874:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb ctime from disk: 1788572541 -> 1788572525 00002000:00010000:3.0:1788572541.912877:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x46:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572541.912929:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360338560:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 259us (359us total) trans 4294967573 rc 0/0 00000100:00100000:3.0:1788572541.912937:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1062 00000100:00100000:3.0:1788572541.913067:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.913188:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360338944 00000100:00100000:2.0:1788572541.913193:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360338944 00000100:00100000:2.0:1788572541.913227:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1063 00000100:00100000:2.0:1788572541.913235:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360338944:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572541.913325:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x47:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572541.913328:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x47:0x0] updating lvb mtime from disk: 0 -> 1788572520 00002000:00010000:2.0:1788572541.913330:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x47:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:2.0:1788572541.913343:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x47:0x0] updating lvb ctime from disk: 1788572541 -> 1788572524 00002000:00010000:2.0:1788572541.913345:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x47:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572541.913374:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360338944:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 139us (306us total) trans 4294967574 rc 0/0 00000100:00100000:2.0:1788572541.913381:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1063 00000100:00100000:2.0:1788572541.913426:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360338816 00000100:00100000:2.0:1788572541.913431:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360338816 00000100:00100000:2.0:1788572541.913471:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4039 00000100:00100000:2.0:1788572541.913479:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360338816:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572541.913491:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572541.913509:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162337 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef784 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.913608:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16233e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.913622:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16233e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.913634:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16233e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572541.913659:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbd:0x0]: returning size 3 00010000:00010000:2.0:1788572541.913756:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162337 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef784 expref: 172 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.913772:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16233e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef784 expref: 171 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.913781:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627e7a95d00) 00010000:00010000:2.0:1788572541.913783:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572541.913788:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:2.0:1788572541.913816:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360338816:12345-192.168.202.40@tcp:101:diff.0 Request processed in 337us (1283us total) trans 4294970327 rc 0/0 00000100:00100000:2.0:1788572541.913823:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4039 00010000:00010000:1.0:1788572541.914227:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914236:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914243:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914249:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.914254:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914261:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914274:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:1.0:1788572541.914278:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.914282:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914288:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1622b9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914308:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914315:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914322:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914327:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572541.914414:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914421:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914430:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:1.0:1788572541.914433:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572541.914436:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.914441:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1622c0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.919603:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.919654:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572541.919686:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360339328 00000100:00100000:2.0:1788572541.919696:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360339328 00000100:00100000:2.0:1788572541.919745:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4040 00000100:00100000:2.0:1788572541.919754:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35900:x1875453360339328:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572541.919826:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.919844:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.919854:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.919893:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.919907:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 65 pid: 23058 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.919921:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2734e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382349312:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572541.920033:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3251480 going to sleep for 11 seconds 00000100:00100000:1.0:1788572541.921280:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360339072 00000100:00100000:1.0:1788572541.921291:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360339072 00000100:00100000:1.0:1788572541.921488:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 303 00000100:00100000:1.0:1788572541.921498:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:2350:x1875453360339072:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572541.921512:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572541.921533:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16233e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef784 expref: 171 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.921555:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffff9627e7a95d00) 00010000:00010000:1.0:1788572541.921560:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572541.921610:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:2350:x1875453360339072:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 113us (2017us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572541.921617:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 303 00000100:00100000:0.0:1788572541.956480:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.956527:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.956549:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.956566:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.956585:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.956627:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572541.956652:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360339712 00000100:00100000:0.0:1788572541.956661:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360339712 00000100:00100000:0.0:1788572541.956707:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360339968 00000100:00100000:0.0:1788572541.956710:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360339968 00000100:00100000:0.0:1788572541.956723:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360340096 00000100:00100000:0.0:1788572541.956726:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360340096 00000100:00100000:0.0:1788572541.956742:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1064 00000100:00100000:0.0:1788572541.956752:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2350:x1875453360339712:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572541.956772:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.956792:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f11b39c/0x85ace67feb162353 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef78b expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.956821:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x42:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:0.0:1788572541.956842:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=890s v=(0 0) 00010000:00010000:0.0:1788572541.956846:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.956856:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f11b39c/0x85ace67feb162353 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef78b expref: 47 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.956875:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f11b39c/0x85ace67feb162353 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef78b expref: 47 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.956889:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094cc00, rc 0) 00000100:00100000:0.0:1788572541.956949:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2350:x1875453360339712:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 198us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.956956:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1064 00000100:00100000:0.0:1788572541.956972:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1065 00000100:00100000:0.0:1788572541.956978:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2349:x1875453360339968:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572541.956984:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.956993:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7510d05/0x85ace67feb16235a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef792 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.957025:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x44:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:0.0:1788572541.957033:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=890s v=(0 0) 00010000:00010000:0.0:1788572541.957036:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x44:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.957042:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7510d05/0x85ace67feb16235a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef792 expref: 47 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.957053:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7510d05/0x85ace67feb16235a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x44:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef792 expref: 47 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.957067:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094da00, rc 0) 00000100:00100000:0.0:1788572541.957084:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2349:x1875453360339968:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 107us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.957090:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1065 00000100:00100000:0.0:1788572541.957101:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1066 00000100:00100000:0.0:1788572541.957107:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+35:2349:x1875453360340096:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572541.957113:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572541.957123:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1378e6c/0x85ace67feb162361 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef799 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572541.957137:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x42:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:0.0:1788572541.957145:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=890s v=(0 0) 00010000:00010000:0.0:1788572541.957148:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x42:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572541.957153:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1378e6c/0x85ace67feb162361 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef799 expref: 36 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.957165:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f1378e6c/0x85ace67feb162361 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x42:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef799 expref: 36 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.957173:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c800, rc 0) 00000100:00100000:0.0:1788572541.957187:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+36:2349:x1875453360340096:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 80us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.957192:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1066 00000100:00100000:1.0:1788572541.957315:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360339584 00000100:00100000:1.0:1788572541.957320:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360339584 00000100:00100000:1.0:1788572541.957358:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4041 00000100:00100000:1.0:1788572541.957365:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35902:x1875453360339584:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572541.957409:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.957423:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.957432:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.957472:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572541.957483:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572541.957497:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd180 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382349440:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572541.957544:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec040780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572541.957729:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360339840 00000100:00100000:3.0:1788572541.957734:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360339840 00002000:00100000:3.0:1788572541.957765:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x45:0x0] (0->4095) req@ffff9627c9a59180 x1875453360339840/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/0 e 0 to 0 dl 1788572552 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572541.957775:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a59180 with x1875453360339840 ext(0->4095) 00002000:00010000:3.0:1788572541.957780:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a59180 00000100:00100000:3.0:1788572541.957803:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 244 00000100:00100000:3.0:1788572541.957810:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2350:x1875453360339840:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572541.957980:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2350:x1875453360339840:12345-192.168.202.40@tcp:3:diff.0 Request processed in 170us (1429us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572541.957988:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 244 00002000:00100000:3.0:1788572541.957994:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x45:0x0] (0->4095) req@ffff9627c9a59180 x1875453360339840/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:202/0 lens 488/4536 e 0 to 0 dl 1788572552 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572541.958026:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a59180 with x1875453360339840 ext(0->4095) 00002000:00010000:3.0:1788572541.958029:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9a59180 00000100:00100000:3.0:1788572541.959335:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2734e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382349312:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572541.959363:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.961226:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360340224 00000100:00100000:2.0:1788572541.961240:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360340224 00010000:00010000:2.0:1788572541.961289:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 67 pid: 23058 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572541.961339:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 958 00000100:00100000:2.0:1788572541.961349:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360340224:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572541.961366:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572541.961373:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 67 pid: 23058 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.961387:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 67 pid: 23058 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.961409:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162185 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x80:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b2 expref: 67 pid: 23058 timeout: 989 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.961427:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:2.0:1788572541.961432:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.961442:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.961450:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572541.962523:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572541.962607:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360340224:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1255us (5974us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572541.962620:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 958 00000001:00080000:3.0:1788572541.965710:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967572 is committed 00000001:00080000:3.0:1788572541.965734:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967569 is committed 00000001:00080000:3.0:1788572541.965752:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967573 is committed 00000001:00080000:3.0:1788572541.965759:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967570 is committed 00010000:00010000:0.0:1788572541.967679:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967697:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.967831:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360337536:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 95376us (95622us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572541.967842:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1060 00010000:00010000:0.0:1788572541.967909:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967917:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967925:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967947:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572541.967951:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967959:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967975:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:0.0:1788572541.967978:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572541.967984:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572541.967991:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16234c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572541.968079:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35900:x1875453360339328:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 48325us (48423us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572541.968089:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4040 00000100:00100000:1.0:1788572541.990371:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd180 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382349440:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572541.990402:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.004812:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.004865:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.004900:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.004976:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.005050:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360340992 00000100:00100000:1.0:1788572542.005060:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360340992 00000100:00100000:1.0:1788572542.005113:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4042 00000100:00100000:2.0:1788572542.005117:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360340608 00000100:00100000:2.0:1788572542.005122:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360340608 00000100:00100000:1.0:1788572542.005124:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35901:x1875453360340992:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.005152:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 882 00000100:00100000:2.0:1788572542.005159:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0e64850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360340608:12345-192.168.202.40@tcp:35:diff.0 00010000:00010000:1.0:1788572542.005186:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.005213:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1621ee lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6e3 expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.005238:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572542.005243:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572542.005274:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.005287:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.005297:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.005517:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0e64850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360340608:12345-192.168.202.40@tcp:35:diff.0 Request processed in 357us (650us total) trans 4294970329 rc 0/0 00000100:00100000:2.0:1788572542.005526:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 882 00000100:00100000:3.0:1788572542.005732:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360340352 00000100:00100000:3.0:1788572542.005738:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360340352 00010000:00010000:3.0:1788572542.005763:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.005800:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 959 00000100:00100000:3.0:1788572542.005807:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360340352:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572542.005818:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572542.005823:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.005835:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.005853:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb162138 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef69d expref: 66 pid: 22536 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.005864:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:3.0:1788572542.005868:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.005875:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572542.005879:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ff109b18 reply state 00000000011f428b transno 4294970328 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.005881:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.005892:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572542.005905:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572542.005926:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360340352:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 120us (1127us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.005934:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 959 00000100:00100000:1.0:1788572542.006036:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35901:x1875453360340992:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 878us (1023us total) trans 4294970328 rc 0/0 00000100:00100000:1.0:1788572542.006045:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4042 00010000:00010000:1.0:1788572542.006070:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006077:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006094:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006102:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006108:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006114:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.006118:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006126:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006136:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:1.0:1788572542.006139:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572542.006143:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.006153:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162368 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.006186:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35902:x1875453360339584:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 48822us (49728us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572542.006192:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4041 00000100:00100000:0.0:1788572542.006387:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360340736 00000100:00100000:0.0:1788572542.006392:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360340736 00000100:00100000:0.0:1788572542.006445:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1067 00000100:00100000:0.0:1788572542.006453:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360340736:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.006494:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.006511:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.006519:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x47:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1788572542.006523:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.006529:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.006560:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.006574:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 13 pid: 13272 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.006589:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382349568:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572542.006663:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00080000:1.0:1788572542.008136:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1788572542.018285:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.018306:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.018314:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.018320:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.018325:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.018334:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.018352:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572542.018356:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.018361:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.018368:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb16236f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.018751:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.019079:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360341120 00000100:00100000:0.0:1788572542.019090:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360341120 00010000:00010000:0.0:1788572542.019128:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 14 pid: 13272 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.019164:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 960 00000100:00100000:0.0:1788572542.019174:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360341120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.019188:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.019226:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 14 pid: 13272 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.019240:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 14 pid: 13272 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.019260:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.019264:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.019270:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.019279:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.019301:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.019352:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360341120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 179us (605us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.019359:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 960 00000100:00100000:2.0:1788572542.019890:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.020200:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.020308:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360341248 00000100:00100000:2.0:1788572542.020315:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360341248 00000100:00100000:2.0:1788572542.020359:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360341376 00000100:00100000:2.0:1788572542.020363:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360341376 00000100:00100000:2.0:1788572542.020413:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1068 00000100:00100000:2.0:1788572542.020423:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360341248:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.020473:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/1,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.020492:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.020500:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x46:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1788572542.020503:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572542.020508:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 4/1,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.020551:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.020565:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 14 pid: 6682 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.020582:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382349696:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.020660:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda300 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.020680:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1069 00000100:00100000:2.0:1788572542.020689:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360341376:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572542.020944:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572542.020948:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb mtime from disk: 1788572541 -> 1788572519 00002000:00010000:2.0:1788572542.020950:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb atime from disk: 0 -> 1788572519 00002000:00010000:2.0:1788572542.020952:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb ctime from disk: 1788572541 -> 1788572525 00002000:00010000:2.0:1788572542.020954:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x47:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572542.020985:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360341376:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 297us (786us total) trans 4294967574 rc 0/0 00000100:00100000:2.0:1788572542.020993:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1069 00000100:00080000:1.0:1788572542.021339:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1788572542.025548:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000933e153b/0x85ace67feb162249 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef722 expref: 14 pid: 13272 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.025573:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382349568:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572542.025603:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025711:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025721:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025767:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025776:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025785:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025792:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572542.025797:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025805:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025824:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.025832:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008806a641/0x85ace67feb162376 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.027118:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360340736:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 20657us (22208us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.027136:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1067 00000100:00100000:1.0:1788572542.032590:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.032701:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382349696:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.032739:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/1,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.032873:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.033156:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360341888 00000100:00100000:0.0:1788572542.033165:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360341888 00000100:00100000:0.0:1788572542.033225:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4043 00000100:00100000:0.0:1788572542.033234:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360341888:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572542.033258:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.033279:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162384 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef7df expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.033404:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16238b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.033420:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16238b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.033431:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16238b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.033446:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbc:0x0]: returning size 3 00010000:00010000:0.0:1788572542.033598:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162384 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7df expref: 174 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.033622:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16238b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7df expref: 173 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.033632:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627ffc51600) 00010000:00010000:0.0:1788572542.033635:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572542.033654:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d600, rc 0) 00000100:00100000:0.0:1788572542.033719:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360341888:12345-192.168.202.40@tcp:101:diff.0 Request processed in 485us (845us total) trans 4294970330 rc 0/0 00000100:00100000:0.0:1788572542.033727:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4043 00000100:00100000:0.0:1788572542.035556:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.035796:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360341632 00000100:00100000:1.0:1788572542.035805:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360341632 00010000:00010000:1.0:1788572542.035846:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 13 pid: 6682 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.035888:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 961 00000100:00100000:1.0:1788572542.035898:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360341632:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.035914:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.035950:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 13 pid: 6682 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.035965:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 13 pid: 6682 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.035989:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb162218 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x46:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef70d expref: 13 pid: 6682 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.036035:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.036040:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.036048:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.036059:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.036359:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.036421:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360341632:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 523us (872us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.036431:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 961 00000100:00100000:1.0:1788572542.040175:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.040250:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360342144 00000100:00100000:0.0:1788572542.040261:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360342144 00000100:00100000:0.0:1788572542.040329:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4044 00000100:00100000:0.0:1788572542.040339:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35901:x1875453360342144:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.040415:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.040431:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.040435:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360341504 00000100:00100000:3.0:1788572542.040439:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360341504 00010000:00010000:0.0:1788572542.040440:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.040480:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 272 00010000:00010000:0.0:1788572542.040483:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.040488:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:2350:x1875453360341504:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572542.040497:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 63 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.040510:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2399500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382349824:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572542.040528:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.040539:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1623a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef7a0 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.040586:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.040596:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.040607:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.040620:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.040626:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.040632:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.040928:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.040940:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.040974:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.040981:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.040989:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.040996:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572542.041000:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.041024:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.041041:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.041049:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d3ee15d6/0x85ace67feb16237d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041315:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041322:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041327:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041343:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.041347:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041353:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041364:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572542.041366:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.041370:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041375:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041385:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041390:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041396:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041401:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.041404:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041409:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.041415:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57a0c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788572542.041421:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x2f385ca3 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.041423:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.041426:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041431:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb1623ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041472:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1623a0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7a0 expref: 174 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041483:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572542.041485:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572542.041489:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1623a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7a0 expref: 174 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.041499:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc -2) 00000100:00100000:3.0:1788572542.041543:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:2350:x1875453360341504:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1055us (8962us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572542.041551:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 272 00000100:00100000:0.0:1788572542.047549:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.047894:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2399500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382349824:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572542.047922:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.049194:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360342400 00000100:00100000:1.0:1788572542.049205:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360342400 00000100:00100000:1.0:1788572542.049280:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4045 00000100:00100000:1.0:1788572542.049290:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35900:x1875453360342400:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.049378:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.049396:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.049409:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.049985:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360341248:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 29560us (30094us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.049996:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1068 00010000:00010000:1.0:1788572542.051085:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.051105:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.051119:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 4/0,1 mode: --/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.051160:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.051173:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 65 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.051187:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382349952:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.051272:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194480 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.051399:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.052675:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360342656 00000100:00100000:1.0:1788572542.052685:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360342656 00002000:00100000:1.0:1788572542.052759:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x42:0x0] (0->4095) req@ffff9627e7bc6a00 x1875453360342656/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:203/0 lens 488/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.052770:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7bc6a00 with x1875453360342656 ext(0->4095) 00002000:00010000:1.0:1788572542.052775:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7bc6a00 00000100:00100000:1.0:1788572542.052803:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 245 00000100:00100000:1.0:1788572542.052812:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2347:x1875453360342656:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572542.053026:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2347:x1875453360342656:12345-192.168.202.40@tcp:3:diff.0 Request processed in 213us (1626us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572542.053034:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 245 00002000:00100000:1.0:1788572542.053040:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x42:0x0] (0->4095) req@ffff9627e7bc6a00 x1875453360342656/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:203/0 lens 488/4536 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.053054:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7bc6a00 with x1875453360342656 ext(0->4095) 00002000:00010000:1.0:1788572542.053058:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7bc6a00 00000100:00080000:3.0:1788572542.053176:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1788572542.077674:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572542.079146:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.079189:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.079248:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.079263:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360342528 00000100:00100000:3.0:1788572542.079271:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343040 00000100:00100000:0.0:1788572542.079272:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360342528 00000100:00100000:3.0:1788572542.079274:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343040 00010000:00010000:0.0:1788572542.079313:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 67 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.079323:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4046 00000100:00100000:3.0:1788572542.079332:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35902:x1875453360343040:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.079333:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360342784 00000100:00100000:0.0:1788572542.079336:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360342784 00000100:00100000:0.0:1788572542.079354:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 962 00000100:00100000:0.0:1788572542.079360:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e51c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360342528:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.079370:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.079376:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 67 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079388:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 67 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079405:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079407:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162169 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x7f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6a4 expref: 67 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079420:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079425:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572542.079429:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079430:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079437:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079442:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079496:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.079542:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e51c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360342528:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 181us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.079548:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 962 00000100:00100000:0.0:1788572542.079593:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 304 00000100:00100000:0.0:1788572542.079600:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:2350:x1875453360342784:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.079609:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.079621:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16238b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7df expref: 173 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079634:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffff9627ffc51600) 00010000:00010000:0.0:1788572542.079637:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.079654:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:2350:x1875453360342784:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 55us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.079660:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 304 00010000:00010000:0.0:1788572542.079683:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079689:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079693:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079703:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079706:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079710:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 4/0,1 mode: --/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079713:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079719:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079725:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.079729:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079736:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.079738:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079750:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:3.0:1788572542.079751:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 66 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079753:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572542.079757:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.079763:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162399 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.079771:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382350080:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572542.079793:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35901:x1875453360342144:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 39456us (40466us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572542.079800:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4044 00000100:00100000:3.0:1788572542.079813:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.080258:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382349952:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.080279:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.082448:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.082502:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.082584:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343296 00000100:00100000:1.0:1788572542.082593:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343296 00000100:00100000:1.0:1788572542.082661:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1070 00000100:00100000:1.0:1788572542.082671:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360343296:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.082681:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382350080:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.082705:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.082715:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.082733:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.082742:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x47:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788572542.082747:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.082752:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.082785:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.082797:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 11 pid: 6679 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.082813:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382350208:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.083156:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3194480 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.083192:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.083225:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343168 00000100:00100000:0.0:1788572542.083230:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343168 00010000:00010000:0.0:1788572542.083253:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 66 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.083275:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343424 00000100:00100000:0.0:1788572542.083278:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343424 00010000:00010000:0.0:1788572542.083290:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.083293:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 963 00000100:00100000:2.0:1788572542.083302:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360343168:12345-192.168.202.40@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1788572542.083308:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 964 00000100:00100000:0.0:1788572542.083314:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360343424:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.083314:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.083320:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.083321:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083324:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.083333:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083334:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083352:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16217e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ab expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.083352:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1621bd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6ce expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083368:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:2.0:1788572542.083371:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:0.0:1788572542.083373:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.083375:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083383:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.083383:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083391:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.083392:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.083408:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572542.083413:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572542.083466:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.083473:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572542.083652:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970331 00000001:00080000:1.0:1788572542.083816:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970333 00000001:00080000:1.0:1788572542.083863:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970334 00000100:00100000:1.0:1788572542.083959:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382350336:0@lo:1000:mdt00_005.0 00010000:00010000:3.0:1788572542.084090:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.084099:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572542.084185:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970332 00000001:00080000:3.0:1788572542.084303:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970336 00000001:00080000:3.0:1788572542.084343:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970337 00000100:00100000:3.0:1788572542.084394:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382350464:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572542.084506:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2349:x1875453360343168:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1203us (2063us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.084515:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 963 00000100:00100000:2.0:1788572542.090991:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.091056:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343680 00000100:00100000:2.0:1788572542.091065:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343680 00000100:00100000:2.0:1788572542.091123:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 883 00000100:00100000:2.0:1788572542.091134:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:35867:x1875453360343680:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572542.091539:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360343680:12345-192.168.202.40@tcp:35:diff.0 Request processed in 406us (557us total) trans 4294970339 rc 0/0 00000100:00100000:2.0:1788572542.091548:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 883 00000100:00100000:0.0:1788572542.093149:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360343424:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 9825us (9947us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.093163:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 964 00000100:00100000:1.0:1788572542.098739:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572542.098795:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b00c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.100589:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382350336 00000100:00100000:2.0:1788572542.100602:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382350336 00000100:00100000:2.0:1788572542.100689:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00080000:1.0:1788572542.100692:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788572542.100699:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:23058:x1875453382350336:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1788572542.100757:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.100806:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.100808:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:23058:x1875453382350336:12345-0@lo:1000:mdt00_005.0 Request processed in 111us (2078us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.100816:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:1.0:1788572542.100825:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.100862:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343808 00000100:00100000:1.0:1788572542.100866:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343808 00010000:00010000:1.0:1788572542.100891:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 12 pid: 6679 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.100918:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 965 00000100:00100000:1.0:1788572542.100925:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360343808:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.100940:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.100960:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 12 pid: 6679 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.100973:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 12 pid: 6679 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101000:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.101025:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.101032:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101042:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101064:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572542.101101:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360343808:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 175us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.101107:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 965 00010000:00010000:1.0:1788572542.101149:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ff5b326/0x85ace67feb162211 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x47:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef706 expref: 11 pid: 6679 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.101162:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382350208:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.101181:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101197:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101204:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101225:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101233:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101241:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101248:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572542.101252:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101261:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101272:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101537:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb1623d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.101597:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360343296:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 18928us (19094us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.101605:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1070 00000100:00100000:1.0:1788572542.101650:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360344192 00000100:00100000:1.0:1788572542.101657:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360344192 00000100:00100000:1.0:1788572542.101693:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4047 00000100:00100000:1.0:1788572542.101702:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360344192:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572542.101723:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.101746:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1623d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef85d expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101843:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1623df lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101857:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1623df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.101867:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1623df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.101878:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbb:0x0]: returning size 3 00010000:00010000:1.0:1788572542.102160:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb1623d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef85d expref: 176 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.102181:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1623df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef85d expref: 175 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.102189:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627ffc51f00) 00010000:00010000:1.0:1788572542.102192:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572542.102195:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90c00, rc 0) 00000100:00100000:1.0:1788572542.102221:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360344192:12345-192.168.202.40@tcp:101:diff.0 Request processed in 520us (1465us total) trans 4294970340 rc 0/0 00000100:00100000:1.0:1788572542.102228:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4047 00000100:00100000:1.0:1788572542.102628:0:23058:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967565 gen 2 00000004:00080000:1.0:1788572542.102635:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967565 committed transno 4294967565 00000100:00100000:1.0:1788572542.102988:0:23058:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627dfae7100 x1875453382348672/t4294967570(4294967570) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1788572557 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572542.103021:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382350336:0@lo:1000:mdt00_005.0 00000001:00080000:1.0:1788572542.103417:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970335 00000100:00100000:1.0:1788572542.103506:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382350592:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1788572542.103559:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.104541:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360343936 00000100:00100000:2.0:1788572542.105225:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360343936 00000100:00100000:2.0:1788572542.105281:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 274 00000100:00100000:2.0:1788572542.105291:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:2349:x1875453360343936:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:2.0:1788572542.108890:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.109027:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.115519:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360344320 00000100:00100000:0.0:1788572542.115535:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360344320 00002000:00100000:0.0:1788572542.115829:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x44:0x0] (0->4095) req@ffff9626c2735f80 x1875453360344320/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:203/0 lens 488/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.115842:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2735f80 with x1875453360344320 ext(0->4095) 00002000:00010000:0.0:1788572542.116161:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2735f80 00000100:00100000:0.0:1788572542.116196:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 246 00000100:00100000:0.0:1788572542.116207:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2349:x1875453360344320:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572542.116591:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2349:x1875453360344320:12345-192.168.202.40@tcp:3:diff.0 Request processed in 386us (7712us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572542.116599:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 246 00002000:00100000:0.0:1788572542.116605:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x44:0x0] (0->4095) req@ffff9626c2735f80 x1875453360344320/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:203/0 lens 488/4536 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.116613:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2735f80 with x1875453360344320 ext(0->4095) 00002000:00010000:0.0:1788572542.116617:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2735f80 00000100:00100000:2.0:1788572542.117226:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360344448 00000100:00100000:2.0:1788572542.117233:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360344448 00000100:00100000:2.0:1788572542.117282:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 305 00000100:00100000:2.0:1788572542.117292:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:2348:x1875453360344448:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.117306:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.117331:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1623df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef85d expref: 177 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.117350:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbb:0x0].0x0 (ffff9627ffc51f00) 00010000:00010000:2.0:1788572542.117357:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.117385:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:2348:x1875453360344448:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 94us (8358us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.117392:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 305 00000100:00100000:0.0:1788572542.117498:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572542.120948:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.120978:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1623ed lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef7e6 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121078:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121093:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121105:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121122:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121129:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121137:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121363:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121371:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121377:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121383:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.121387:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121395:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121409:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572542.121412:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.121417:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121424:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1623f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121438:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121445:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121452:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121469:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.121472:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121479:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121486:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x1dd915a3 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.121489:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.121492:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121498:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1623fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121534:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1623ed lrc: 1/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7e6 expref: 176 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121545:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:3.0:1788572542.121547:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572542.121551:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1623ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef7e6 expref: 176 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.121561:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc000, rc -2) 00000100:00100000:3.0:1788572542.121887:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:2349:x1875453360343936:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 16356us (20819us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572542.121901:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 274 00000100:00100000:2.0:1788572542.122038:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360344704 00000100:00100000:2.0:1788572542.122047:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360344704 00000100:00100000:2.0:1788572542.122108:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4048 00000100:00100000:2.0:1788572542.122117:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35901:x1875453360344704:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.122178:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122190:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122199:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122422:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122434:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 72 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122442:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122488:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 72 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.122504:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 72 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.122519:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382350720:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.123852:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda180 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.128266:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.128348:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.128371:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4f14780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.128929:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360345088 00000100:00100000:0.0:1788572542.128940:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360345088 00010000:00010000:0.0:1788572542.128981:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 73 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.129050:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 966 00000100:00100000:0.0:1788572542.129061:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2350:x1875453360345088:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.129077:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.129085:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 73 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.129098:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 73 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.129119:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:0.0:1788572542.129124:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.129135:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.129143:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.129163:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.129216:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2350:x1875453360345088:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 157us (868us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.129224:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 966 00000100:00100000:0.0:1788572542.129251:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382350592 00000100:00100000:0.0:1788572542.129256:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382350592 00000100:00100000:0.0:1788572542.129288:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:0.0:1788572542.129295:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:23058:x1875453382350592:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572542.129562:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:23058:x1875453382350592:12345-0@lo:1000:mdt00_005.0 Request processed in 267us (26001us total) trans 4294967575 rc 0/0 00000100:00100000:0.0:1788572542.129569:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:2.0:1788572542.129839:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360344960 00000100:00100000:2.0:1788572542.129845:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360344960 00000100:00100000:2.0:1788572542.129887:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 884 00000100:00100000:2.0:1788572542.129895:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:35867:x1875453360344960:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572542.130214:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360344960:12345-192.168.202.40@tcp:35:diff.0 Request processed in 318us (1957us total) trans 4294970343 rc 0/0 00000100:00100000:2.0:1788572542.130221:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 884 00010000:00010000:2.0:1788572542.130269:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16218c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef6b9 expref: 72 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.130287:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382350720:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.130310:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.130319:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.130325:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572542.130422:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970342 00000001:00080000:2.0:1788572542.130596:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970344 00000001:00080000:2.0:1788572542.130731:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970345 00000100:00100000:2.0:1788572542.130822:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382350848:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572542.130867:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.130904:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382350848 00000100:00100000:2.0:1788572542.130909:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382350848 00000100:00100000:2.0:1788572542.130945:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:2.0:1788572542.130952:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382350848:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572542.131032:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382350848:12345-0@lo:1000:mdt00_004.0 Request processed in 80us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.131039:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00080000:2.0:1788572542.131059:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572542.131071:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c99d8a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.131085:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967573 gen 2 00000004:00080000:2.0:1788572542.131090:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967567 committed transno 4294967573 00000004:00080000:2.0:1788572542.131104:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967568 committed transno 4294967573 00000004:00080000:2.0:1788572542.131117:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c12eea00, transno 4294967571 00000004:00080000:2.0:1788572542.131126:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c1bcc000, transno 4294967572 00000004:00080000:2.0:1788572542.131129:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff962801f92300, transno 4294967573 00000100:00100000:2.0:1788572542.131135:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382350848:0@lo:1000:mdt00_004.0 00000004:00080000:2.0:1788572542.131413:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:1.0:1788572542.131666:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382350592:0@lo:1000:mdt00_005.0 00010000:00010000:1.0:1788572542.131695:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131704:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131712:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131718:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.131722:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131729:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131743:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:1.0:1788572542.131746:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.131750:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 2/0,0 mode: --/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131756:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1623bc lrc: 0/0,0 mode: --/EX res: [0x200000402:0x80:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131784:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131792:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131799:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131803:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d5e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.131806:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131815:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131823:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627ffc16500) 00010000:00010000:1.0:1788572542.131825:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.131828:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.131834:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5e00/0x85ace67feb15f0bd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132050:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132063:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132072:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132080:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132085:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132092:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132097:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.132100:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132106:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132113:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627ffc16500) 00010000:00010000:1.0:1788572542.132116:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.132118:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132124:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb162410 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x86:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132133:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132139:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132145:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132150:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.132153:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132159:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132166:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627ffc16500) 00010000:00010000:1.0:1788572542.132169:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572542.132172:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.132182:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1623b5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.132182:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572542.132216:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c63cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.132269:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382350464 00000100:00100000:0.0:1788572542.132276:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382350464 00000100:00100000:0.0:1788572542.132313:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1788572542.132321:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:9040:x1875453382350464:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572542.132338:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35900:x1875453360342400:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 83048us (84798us total) trans 4294970341 rc 0/0 00000100:00100000:1.0:1788572542.132346:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4045 00000020:00080000:1.0:1788572542.132350:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000001:00080000:2.0:1788572542.132472:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970346 00000100:00100000:2.0:1788572542.132533:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382350976:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572542.135951:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:9040:x1875453382350464:12345-0@lo:1000:mdt00_003.0 Request processed in 1326us (1463us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.135969:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:1.0:1788572542.140107:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382350464:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1788572542.140583:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970338 00000100:00100000:1.0:1788572542.140665:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5968e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382351104:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572542.140730:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572542.140771:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.144081:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382351104 00000100:00100000:2.0:1788572542.144097:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382351104 00000100:00100000:2.0:1788572542.144559:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1788572542.144572:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:9040:x1875453382351104:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572542.144694:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.144966:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:9040:x1875453382351104:12345-0@lo:1000:mdt00_003.0 Request processed in 398us (4235us total) trans 4294967576 rc 0/0 00000100:00100000:2.0:1788572542.144975:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:1.0:1788572542.145057:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5968e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382351104:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1788572542.145101:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145110:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145118:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145123:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.145128:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145135:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145152:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:1.0:1788572542.145155:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.145159:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 2/0,0 mode: --/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145165:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1623ca lrc: 0/0,0 mode: --/EX res: [0x200000402:0x7e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145194:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145200:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145205:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145222:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d5a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.145224:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145231:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145238:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627ffc16b00) 00010000:00010000:1.0:1788572542.145240:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.145243:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145249:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5a00/0x85ace67feb15f16c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145276:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145290:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145300:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145307:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145312:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145317:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145322:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.145326:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145331:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145339:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627ffc16b00) 00010000:00010000:1.0:1788572542.145342:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.145345:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145350:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162417 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x87:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145361:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145367:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145372:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145377:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.145380:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145386:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145393:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627ffc16b00) 00010000:00010000:1.0:1788572542.145395:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572542.145398:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.145403:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1623c3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.145481:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360345344 00000100:00100000:2.0:1788572542.145487:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360345344 00000100:00100000:2.0:1788572542.145525:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 885 00000100:00100000:2.0:1788572542.145533:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35900:x1875453360345344:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.145542:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35902:x1875453360343040:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 66211us (66294us total) trans 4294970348 rc 0/0 00000100:00100000:1.0:1788572542.145549:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4046 00000020:00080000:1.0:1788572542.145553:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572542.146084:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572542.146121:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c99d8540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.146139:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382350976 00000100:00100000:1.0:1788572542.146144:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382350976 00000100:00100000:1.0:1788572542.146179:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:1.0:1788572542.146187:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382350976:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572542.146344:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382350976:12345-0@lo:1000:mdt00_004.0 Request processed in 157us (261us total) trans 4294967575 rc 0/0 00000100:00100000:1.0:1788572542.146351:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:1.0:1788572542.146388:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382350976:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572542.146416:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146424:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146431:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146436:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.146439:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146446:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146464:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627d599a000) 00010000:00010000:1.0:1788572542.146467:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.146471:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146477:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162409 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x7f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146506:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146513:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146519:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146524:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d5200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.146526:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146532:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146540:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572542.146543:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.146546:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146551:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5200/0x85ace67feb15f0f5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146606:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146616:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146625:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146631:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146638:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146644:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146648:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.146651:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146657:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146664:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572542.146666:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.146668:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146673:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16241e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146683:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146689:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146695:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146701:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.146704:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146710:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146717:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572542.146719:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572542.146722:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.146728:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162402 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.146847:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35901:x1875453360344704:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 24730us (29348us total) trans 4294970347 rc 0/0 00000100:00100000:1.0:1788572542.146856:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4048 00000020:00080000:1.0:1788572542.146859:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1788572542.147195:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572542.147199:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572542.147230:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 133 count 101 00000040:00080000:3.0:1788572542.147236:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:3.0:1788572542.147586:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc4000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382351232:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572542.147659:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628008010c0 going to sleep for 11 seconds 00000004:00080000:2.0:1788572542.147837:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x80:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572542.148744:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970349 00010000:00010000:2.0:1788572542.148822:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.148841:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.148932:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.148944:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.148949:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.148956:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.148979:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788572542.149089:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572542.149097:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.149108:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572542.149112:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.149120:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.149133:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x80:0x0].0x0 (ffff9627e7a95c00) 00010000:00010000:0.0:1788572542.149135:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572542.149140:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.149146:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162425 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x80:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.149161:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572542.149163:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572542.149165:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572542.149279:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35900:x1875453360345344:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 3742us (4582us total) trans 4294970350 rc 0/0 00000100:00100000:2.0:1788572542.149290:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 885 00000020:00080000:2.0:1788572542.149297:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788572542.157793:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.157856:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360345856 00000100:00100000:0.0:1788572542.157867:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360345856 00000100:00100000:2.0:1788572542.158045:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.158079:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360345600 00000100:00100000:1.0:1788572542.158084:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360345600 00000100:00100000:1.0:1788572542.158126:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4049 00000100:00100000:1.0:1788572542.158137:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360345600:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572542.158166:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.158190:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16242c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef864 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.158306:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162433 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.158325:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162433 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.158338:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162433 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.158350:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xba:0x0]: returning size 3 00010000:00010000:1.0:1788572542.158506:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16242c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef864 expref: 178 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.158524:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162433 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef864 expref: 177 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.158534:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627ffc51e00) 00010000:00010000:1.0:1788572542.158537:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572542.158541:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:1.0:1788572542.158601:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:35867:x1875453360345600:12345-192.168.202.40@tcp:101:diff.0 Request processed in 465us (555us total) trans 4294970351 rc 0/0 00000100:00100000:1.0:1788572542.158608:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4049 00000100:00100000:1.0:1788572542.158676:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.159258:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.163495:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.163528:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.163548:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.163571:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.164147:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.164185:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360346880 00000100:00100000:1.0:1788572542.164196:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360346880 00000100:00100000:1.0:1788572542.164325:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 306 00000100:00100000:1.0:1788572542.164337:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e50c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:2348:x1875453360346880:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.164353:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.164377:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162433 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef864 expref: 177 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.164397:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffff9627ffc51e00) 00010000:00010000:1.0:1788572542.164402:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572542.164452:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e50c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:2348:x1875453360346880:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 118us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.164471:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 306 00000100:00100000:1.0:1788572542.164528:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360346368 00000100:00100000:1.0:1788572542.164532:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360346368 00000100:00100000:1.0:1788572542.164558:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360346496 00000100:00100000:1.0:1788572542.164561:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360346496 00000100:00100000:1.0:1788572542.164577:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360346624 00000100:00100000:1.0:1788572542.164580:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360346624 00000100:00100000:1.0:1788572542.164597:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4050 00000100:00100000:1.0:1788572542.164604:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35906:x1875453360346368:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572542.164636:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x8a:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572542.164718:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35906:x1875453360346368:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 114us (1232us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.164725:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4050 00000100:00100000:1.0:1788572542.164738:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4051 00000100:00100000:1.0:1788572542.164744:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35905:x1875453360346496:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572542.164756:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x84:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572542.164801:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35905:x1875453360346496:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 56us (1272us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.164807:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4051 00000100:00100000:1.0:1788572542.164820:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4052 00000100:00100000:1.0:1788572542.164826:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35907:x1875453360346624:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572542.164840:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x85:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572542.165028:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35907:x1875453360346624:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 201us (1478us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.165036:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4052 00000100:00100000:1.0:1788572542.165082:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc4000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382351232:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572542.165771:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360346240 00000100:00100000:0.0:1788572542.165780:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360346240 00000100:00100000:0.0:1788572542.165816:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 886 00000100:00100000:0.0:1788572542.165823:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35901:x1875453360345856:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572542.166152:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x7f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572542.166196:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970352 00010000:00010000:0.0:1788572542.166241:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166258:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166270:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166279:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166286:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166293:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166316:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572542.166525:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35901:x1875453360345856:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 700us (8740us total) trans 4294970353 rc 0/0 00000100:00100000:0.0:1788572542.166533:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 886 00000020:00080000:0.0:1788572542.166538:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788572542.166554:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 887 00000100:00100000:0.0:1788572542.166563:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35902:x1875453360346240:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572542.166819:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x7e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572542.166848:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970354 00010000:00010000:0.0:1788572542.166882:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166892:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166899:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166911:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166916:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166923:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.166942:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1788572542.166981:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35902:x1875453360346240:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 419us (7724us total) trans 4294970355 rc 0/0 00000100:00100000:0.0:1788572542.166987:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 887 00000020:00080000:0.0:1788572542.166990:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788572542.167035:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360346752 00000100:00100000:0.0:1788572542.167041:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360346752 00002000:00100000:0.0:1788572542.167085:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x42:0x0] (0->4095) req@ffff9627f5968a80 x1875453360346752/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:203/0 lens 488/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.167095:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5968a80 with x1875453360346752 ext(0->4095) 00002000:00010000:0.0:1788572542.167099:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f5968a80 00000100:00100000:0.0:1788572542.167118:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 247 00000100:00100000:0.0:1788572542.167125:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+36:2349:x1875453360346752:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572542.167267:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+36:2349:x1875453360346752:12345-192.168.202.40@tcp:3:diff.0 Request processed in 142us (3695us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572542.167274:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 247 00002000:00100000:0.0:1788572542.167280:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x42:0x0] (0->4095) req@ffff9627f5968a80 x1875453360346752/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:203/0 lens 488/4536 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.167287:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5968a80 with x1875453360346752 ext(0->4095) 00002000:00010000:0.0:1788572542.167290:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f5968a80 00010000:00010000:0.0:1788572542.167307:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1788572542.167314:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.167320:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572542.167325:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.168092:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360345984 00000100:00100000:3.0:1788572542.168099:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360345984 00000100:00100000:3.0:1788572542.168136:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 280 00000100:00100000:3.0:1788572542.168143:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:2350:x1875453360345984:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00080000:2.0:1788572542.168321:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1788572542.169409:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169431:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7f:0x0].0x0 (ffff9627e7451b00) 00010000:00010000:0.0:1788572542.169434:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572542.169441:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169449:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162441 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169482:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572542.169487:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169492:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572542.169495:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169502:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169510:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x7e:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:0.0:1788572542.169513:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572542.169515:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169521:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162448 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x7e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.169531:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572542.169532:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572542.169534:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572542.170239:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.170302:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572542.170327:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572542.170506:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360347008 00000100:00100000:2.0:1788572542.170516:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360347008 00000100:00100000:2.0:1788572542.170561:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360347136 00000100:00100000:2.0:1788572542.170565:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360347136 00000100:00100000:1.0:1788572542.170570:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4053 00000100:00100000:1.0:1788572542.170580:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35906:x1875453360347008:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.170585:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4054 00000100:00100000:2.0:1788572542.170590:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35905:x1875453360347136:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.170644:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.170655:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170663:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170674:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170689:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170700:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170706:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170734:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170743:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.170748:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.170792:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8a:0x0]: returning size 3 00000004:00200000:1.0:1788572542.170796:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572542.170811:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174202:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174228:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174259:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174267:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174275:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174307:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174314:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174320:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.174438:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x84:0x0]: returning size 3 00000004:00200000:2.0:1788572542.174443:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572542.174472:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174484:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174496:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174502:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174508:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.174513:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174520:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174533:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572542.174536:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.174540:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174547:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162456 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174562:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174568:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174574:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174579:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.174582:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174588:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174595:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x250ed659 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.174598:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.174601:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174606:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16246b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x250ed659 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174622:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174628:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174634:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.174639:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.174642:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.175440:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.175453:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627ffc16c00) 00010000:00010000:2.0:1788572542.175466:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.175471:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.175478:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162472 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.175629:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35905:x1875453360347136:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 5032us (5320us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.175642:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4054 00010000:00010000:2.0:1788572542.178941:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.178971:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162479 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef86b expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179063:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179077:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179089:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179105:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179112:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179118:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179320:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179345:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179354:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179359:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.179364:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179372:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179385:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572542.179388:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.179393:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179399:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162480 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179411:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179418:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179424:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179429:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.179433:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179438:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179446:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0xa16187a3 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572542.179448:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.179451:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179489:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb162487 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179525:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162479 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef86b expref: 178 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179536:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:2.0:1788572542.179538:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572542.179542:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162479 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef86b expref: 178 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.179551:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc -2) 00000100:00100000:2.0:1788572542.179637:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:2350:x1875453360345984:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 11493us (20959us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572542.179657:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 280 00010000:00010000:1.0:1788572542.182738:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182757:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182765:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182770:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.182775:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182784:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182802:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572542.182805:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.182811:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182818:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16244f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182834:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182840:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182859:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182865:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.182868:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182874:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182881:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xecee9641 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572542.182884:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.182887:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182897:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16245d lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xecee9641 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182916:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182922:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182928:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182933:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.182936:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182943:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182950:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d1866600) 00010000:00010000:1.0:1788572542.182953:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.182956:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.182961:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162464 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.183174:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35906:x1875453360347008:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 12590us (12934us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.183174:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.183185:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4053 00000100:00100000:0.0:1788572542.183215:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360347264 00000100:00100000:0.0:1788572542.183223:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360347264 00000100:00100000:0.0:1788572542.183265:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4055 00000100:00100000:0.0:1788572542.183274:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35907:x1875453360347264:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.183275:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572542.183352:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183365:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183375:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183389:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183400:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183406:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183432:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183440:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183447:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.183494:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x85:0x0]: returning size 3 00000004:00200000:0.0:1788572542.183499:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572542.183513:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183523:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183530:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183535:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183540:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.183543:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183550:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183558:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572542.183560:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.183563:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183569:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16248e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183583:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183589:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183594:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183598:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.183600:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183605:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183612:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xd3a16398 (ffff9627e7451100) 00010000:00010000:0.0:1788572542.183614:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.183616:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183626:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162495 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd3a16398 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183639:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183874:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183882:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183887:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.183889:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183895:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183904:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627ffc16900) 00010000:00010000:0.0:1788572542.183907:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.183910:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.183915:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16249c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.184144:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360347520 00000100:00100000:1.0:1788572542.184149:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360347520 00000100:00100000:1.0:1788572542.184185:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 888 00000100:00100000:1.0:1788572542.184193:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:35867:x1875453360347520:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572542.188508:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35907:x1875453360347264:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 5221us (5320us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.188523:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4055 00000100:00100000:1.0:1788572542.190313:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd2850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:35867:x1875453360347520:12345-192.168.202.40@tcp:35:diff.0 Request processed in 6110us (7026us total) trans 4294970356 rc 0/0 00000100:00100000:1.0:1788572542.190327:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 888 00000100:00100000:3.0:1788572542.194144:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.194396:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360347776 00000100:00100000:1.0:1788572542.194407:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360347776 00000100:00100000:1.0:1788572542.194470:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4056 00000100:00100000:1.0:1788572542.194479:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35905:x1875453360347776:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.194508:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.194528:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1624a3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef952 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194634:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194650:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194661:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.194672:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x84:0x0]: returning size 3 00010000:00010000:1.0:1788572542.194771:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194779:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194786:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194797:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.194801:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194809:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194822:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627ffc16c00) 00010000:00010000:1.0:1788572542.194825:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572542.194830:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194836:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1624aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194878:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1624a3 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef952 expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194889:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572542.194892:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572542.194895:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1624a3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef952 expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.194905:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc 0) 00000100:00100000:1.0:1788572542.194978:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35905:x1875453360347776:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 501us (844us total) trans 4294970357 rc 301/301 00000100:00100000:1.0:1788572542.194986:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4056 00000100:00100000:1.0:1788572542.198259:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.198328:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360348032 00000100:00100000:2.0:1788572542.198347:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360348032 00000100:00100000:2.0:1788572542.198406:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4057 00000100:00100000:2.0:1788572542.198417:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35906:x1875453360348032:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.198438:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.198467:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1624b8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef959 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198579:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198594:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198606:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.198618:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8a:0x0]: returning size 3 00010000:00010000:2.0:1788572542.198897:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198911:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198918:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198924:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.198928:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198937:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198950:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d1866600) 00010000:00010000:2.0:1788572542.198954:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572542.198959:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.198965:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb1624bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.199050:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1624b8 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef959 expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.199061:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572542.199064:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572542.199068:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1624b8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef959 expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.199077:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:3.0:1788572542.199099:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.199135:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35906:x1875453360348032:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 719us (886us total) trans 4294970358 rc 301/301 00000100:00100000:2.0:1788572542.199142:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4057 00000100:00100000:2.0:1788572542.199162:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360348288 00000100:00100000:2.0:1788572542.199166:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360348288 00000100:00100000:2.0:1788572542.199199:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4058 00000100:00100000:2.0:1788572542.199207:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:35867:x1875453360348288:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572542.199219:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.199230:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef960 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.199274:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00010000:00010000:2.0:1788572542.199345:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624cd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef960 expref: 179 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.199354:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572542.199357:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572542.199361:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef960 expref: 179 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.199370:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:2.0:1788572542.199399:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360348288:12345-192.168.202.40@tcp:101:diff.0 Request processed in 192us (301us total) trans 4294970359 rc 301/301 00000100:00100000:2.0:1788572542.199406:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4058 00000100:00100000:3.0:1788572542.202495:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.202568:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360348416 00000100:00100000:1.0:1788572542.202577:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360348416 00000100:00100000:1.0:1788572542.202637:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4059 00000100:00100000:1.0:1788572542.202647:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35905:x1875453360348416:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572542.202678:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x84:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572542.202791:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35905:x1875453360348416:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 145us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.202799:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4059 00000100:00100000:1.0:1788572542.205869:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.205972:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360348544 00000100:00100000:2.0:1788572542.205982:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360348544 00000100:00100000:2.0:1788572542.206073:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4060 00000100:00100000:2.0:1788572542.206083:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35906:x1875453360348544:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572542.206116:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x8a:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572542.206239:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35906:x1875453360348544:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 157us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.206247:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4060 00000100:00100000:0.0:1788572542.207027:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.207055:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360348800 00000100:00100000:2.0:1788572542.207060:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360348800 00000100:00100000:2.0:1788572542.207097:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4061 00000100:00100000:2.0:1788572542.207106:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35907:x1875453360348800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.207134:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.207155:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1624db lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef967 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207256:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207270:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207281:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.207292:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x85:0x0]: returning size 3 00010000:00010000:2.0:1788572542.207787:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207796:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207801:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207807:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.207812:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207819:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207832:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627ffc16900) 00010000:00010000:2.0:1788572542.207835:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572542.207840:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207850:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207895:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1624db lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef967 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207906:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572542.207909:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572542.207912:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1624db lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef967 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.207920:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:2.0:1788572542.207986:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35907:x1875453360348800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 880us (983us total) trans 4294970360 rc 301/301 00000100:00100000:2.0:1788572542.207993:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4061 00000100:00100000:0.0:1788572542.210829:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.210933:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.210965:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.211039:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360349184 00000100:00100000:1.0:1788572542.211040:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360349312 00000100:00100000:1.0:1788572542.211048:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360349312 00000100:00100000:2.0:1788572542.211048:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360349184 00000100:00100000:1.0:1788572542.211102:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4062 00000100:00100000:1.0:1788572542.211113:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35907:x1875453360349312:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572542.211150:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x85:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572542.211258:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35907:x1875453360349312:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 146us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.211265:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4062 00000100:00100000:3.0:1788572542.211430:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360349056 00000100:00100000:3.0:1788572542.211435:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360349056 00000100:00100000:3.0:1788572542.211484:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 889 00000100:00100000:3.0:1788572542.211492:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360349056:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572542.211680:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360349056:12345-192.168.202.40@tcp:35:diff.0 Request processed in 189us (859us total) trans 4294970361 rc 0/0 00000100:00100000:3.0:1788572542.211691:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 889 00000100:00100000:0.0:1788572542.215705:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.215832:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360349440 00000100:00100000:1.0:1788572542.215842:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360349440 00000100:00100000:2.0:1788572542.215882:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4063 00000100:00100000:2.0:1788572542.215900:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35906:x1875453360349184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.215923:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.215953:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1624f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef96e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216041:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216057:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216068:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216084:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216090:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216098:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216970:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162505 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216984:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162505 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.216992:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162505 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.217044:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:2.0:1788572542.217048:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572542.217905:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162505 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.217923:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.217934:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.217949:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.217956:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.217960:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.217971:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.217986:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572542.217989:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.217994:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218087:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb1624f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218160:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218180:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218188:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218192:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.218198:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218208:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218220:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627e7a95f00) 00010000:00010000:2.0:1788572542.218222:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.218226:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218231:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb1624fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218271:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb1624f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef96e expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218285:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162505 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef96e expref: 84 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.218292:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572542.218299:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572542.218302:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:2.0:1788572542.218485:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35906:x1875453360349184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2590us (7550us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.218497:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4063 00000100:00100000:2.0:1788572542.219679:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4064 00000100:00100000:2.0:1788572542.219696:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360349440:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572542.219719:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.219736:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16250c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef975 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.219800:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.219814:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.219824:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.219840:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220329:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220340:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220673:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220684:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220692:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.220730:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb7:0x0]: returning size 3 00000004:00200000:2.0:1788572542.220735:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572542.220754:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220771:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220779:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220785:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220791:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220798:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.220802:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220809:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220821:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572542.220824:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.220828:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.220834:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb162513 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222635:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222654:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222662:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222668:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.222671:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222680:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222694:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x9beb5ca3 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.222697:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.222702:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222709:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16251a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222767:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb16250c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 181 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222783:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 180 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.222790:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:2.0:1788572542.222792:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572542.222795:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90200, rc 0) 00000100:00100000:2.0:1788572542.222883:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:35867:x1875453360349440:12345-192.168.202.40@tcp:101:diff.0 Request processed in 3193us (7250us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.222894:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4064 00000100:00100000:2.0:1788572542.224422:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.224474:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.224502:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360349696 00000100:00100000:2.0:1788572542.224510:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360349696 00000100:00100000:2.0:1788572542.224547:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360349824 00000100:00100000:2.0:1788572542.224550:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360349824 00000100:00100000:2.0:1788572542.224569:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4065 00000100:00100000:2.0:1788572542.224578:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35906:x1875453360349696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.224598:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.224601:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572542.224621:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162505 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef96e expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.224637:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572542.224642:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572542.224826:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4066 00000100:00100000:1.0:1788572542.224837:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35905:x1875453360349824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.224846:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.224861:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162528 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef983 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.224923:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.224950:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.224960:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.224975:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.224981:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.224987:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225306:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb16253d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225316:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb16253d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225323:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb16253d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.225597:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:1.0:1788572542.225602:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572542.225621:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb16253d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225632:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225766:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225776:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225781:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.225785:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225793:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225805:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572542.225808:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.225812:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225819:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb16252f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225831:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225837:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225843:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225848:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.225852:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225858:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225865:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572542.225867:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.225870:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225876:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb162536 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225911:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb162528 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef983 expref: 86 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225982:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb16253d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef983 expref: 85 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.225992:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.225994:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572542.225997:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051800, rc 0) 00000100:00100000:1.0:1788572542.226068:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35905:x1875453360349824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1232us (1593us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.226076:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4066 00010000:00010000:2.0:1788572542.226170:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162544 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef97c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572542.226774:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x48:0x0], index 0: 0 00010000:00010000:2.0:1788572542.227082:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16254b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.227093:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16254b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.227102:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16254b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.227162:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162544 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef97c expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.227177:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16254b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef97c expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.227185:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572542.227187:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572542.227190:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:2.0:1788572542.227224:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35906:x1875453360349696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2647us (2808us total) trans 4294970362 rc 0/0 00000100:00100000:2.0:1788572542.227231:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4065 00000100:00100000:3.0:1788572542.227618:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.228195:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360350080 00000100:00100000:1.0:1788572542.228204:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360350080 00000100:00100000:1.0:1788572542.228273:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4067 00000100:00100000:1.0:1788572542.228283:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:35867:x1875453360350080:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572542.228299:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.228313:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162559 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef98a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228831:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228844:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228854:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.228864:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb7:0x0]: returning size 3 00010000:00010000:1.0:1788572542.228954:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228961:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228967:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228973:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.228978:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228986:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.228997:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:1.0:1788572542.229000:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572542.229029:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.229035:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52000/0x85ace67feb162560 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.229072:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162559 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef98a expref: 182 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.229082:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572542.229085:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572542.229089:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb162559 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef98a expref: 182 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.229099:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050400, rc 0) 00000100:00100000:1.0:1788572542.229145:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360350080:12345-192.168.202.40@tcp:101:diff.0 Request processed in 863us (1527us total) trans 4294970363 rc 301/301 00000100:00100000:1.0:1788572542.229152:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4067 00000100:00100000:3.0:1788572542.231087:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.231175:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360350336 00000100:00100000:1.0:1788572542.231184:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360350336 00000100:00100000:1.0:1788572542.231242:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4068 00000100:00100000:1.0:1788572542.231252:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35906:x1875453360350336:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.231304:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572542.231326:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16254b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef97c expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231342:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:1.0:1788572542.231358:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb16253d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef983 expref: 85 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231368:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.231371:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572542.231391:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231403:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231413:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231428:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231440:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231447:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231667:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231676:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231683:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231689:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd52600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.231693:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231701:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231710:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.231713:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.231716:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231722:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd52600/0x85ace67feb16256e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231731:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231736:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231742:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231747:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.231750:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231755:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231762:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x5d702df3 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572542.231765:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.231768:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.231778:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb162575 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x5d702df3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.231857:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35906:x1875453360350336:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 605us (777us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572542.231865:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4068 00000100:00100000:0.0:1788572542.236322:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.236404:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.237056:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.239929:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360350464 00000100:00100000:2.0:1788572542.239944:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360350464 00000100:00100000:2.0:1788572542.240027:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360350592 00000100:00100000:2.0:1788572542.240030:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360350592 00000100:00100000:2.0:1788572542.240060:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360350848 00000100:00100000:2.0:1788572542.240063:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360350848 00000100:00100000:1.0:1788572542.240087:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4069 00000100:00100000:1.0:1788572542.240100:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360350464:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572542.240126:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.240148:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef991 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.240517:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.240531:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 4/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 182 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.240554:0:5839:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.240569:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 3/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 182 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.240583:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:1.0:1788572542.240587:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.240591:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 3/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 182 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.240599:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53400, rc 0) 00000100:00100000:1.0:1788572542.240663:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360350464:12345-192.168.202.40@tcp:101:diff.0 Request processed in 566us (4351us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.243143:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4069 00000100:00100000:2.0:1788572542.243160:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4070 00000100:00100000:1.0:1788572542.243175:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4071 00000100:00100000:2.0:1788572542.243180:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35906:x1875453360350592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.243182:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35905:x1875453360350848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.243198:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.243199:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.243221:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162583 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef998 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.243259:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.243278:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.243285:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162591 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef99f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.243289:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572542.243682:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x48:0x0], index 1: 0 00010000:00010000:1.0:1788572542.243872:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb162598 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.243883:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb162598 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.243891:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb162598 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.243959:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb162591 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef99f expref: 89 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.243978:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb162598 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef99f expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.243992:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572542.243995:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572542.243999:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:1.0:1788572542.244089:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35905:x1875453360350848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 907us (7033us total) trans 4294970364 rc 0/0 00000100:00100000:1.0:1788572542.244097:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4071 00010000:00010000:2.0:1788572542.246043:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162583 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 88 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.246070:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.246078:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.246081:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572542.246084:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:2.0:1788572542.246150:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35906:x1875453360350592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2979us (9744us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.246159:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4070 00000100:00100000:1.0:1788572542.249226:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.249259:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360350976 00000100:00100000:1.0:1788572542.249268:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360350976 00000100:00100000:1.0:1788572542.249314:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4072 00000100:00100000:1.0:1788572542.249323:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35905:x1875453360350976:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.249344:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.249362:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb1625a6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9a6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249425:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249440:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249451:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249476:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249482:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249490:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249763:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1625bb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249773:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1625bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249780:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1625bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.249806:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:1.0:1788572542.249809:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572542.249826:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1625bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249837:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249844:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249850:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249856:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.249860:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249867:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249880:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572542.249883:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.249887:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249893:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1625ad lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249905:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249910:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249916:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249921:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.249924:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249943:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249950:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627d0ee2300) 00010000:00010000:1.0:1788572542.249952:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.249954:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249960:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb1625b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.249992:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb1625a6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9a6 expref: 89 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.250025:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1625bb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9a6 expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.250032:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.250034:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572542.250037:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050c00, rc 0) 00000100:00100000:1.0:1788572542.250085:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35905:x1875453360350976:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 763us (879us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.250092:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4072 00000100:00100000:1.0:1788572542.252594:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.253897:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.253931:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360351232 00000100:00100000:1.0:1788572542.253940:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360351232 00000100:00100000:1.0:1788572542.253988:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360351488 00000100:00100000:1.0:1788572542.253992:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360351488 00000100:00100000:1.0:1788572542.254033:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4073 00000100:00100000:1.0:1788572542.254043:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35907:x1875453360351232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.254066:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.254085:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1625c2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9ad expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572542.254517:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x49:0x0], index 0: 0 00010000:00010000:1.0:1788572542.254708:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1625c9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.254723:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1625c9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.254734:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1625c9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.254758:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4074 00000100:00100000:0.0:1788572542.254779:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35906:x1875453360351488:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.254789:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1625c2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9ad expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.254807:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1625c9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9ad expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.254815:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627d0ee2200) 00010000:00010000:1.0:1788572542.254818:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572542.254821:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00010000:00010000:0.0:1788572542.254829:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.254837:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.254844:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.254874:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35907:x1875453360351232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 833us (2289us total) trans 4294970365 rc 0/0 00000100:00100000:1.0:1788572542.254882:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4073 00000040:00080000:0.0:1788572542.255152:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].60, 88 off12992 00000004:00010000:0.0:1788572542.255217:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f117c84b reply state 000000006e853011 transno 4294970366 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.255226:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572542.255293:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35906:x1875453360351488:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 527us (1400us total) trans 4294970366 rc 0/0 00000100:00100000:0.0:1788572542.255300:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4074 00010000:00010000:2.0:1788572542.259292:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.259304:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572542.259309:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.259317:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.259322:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.259328:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.259347:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.259364:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.259368:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572542.259373:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.259381:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1625d7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.259517:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360351616 00000100:00100000:3.0:1788572542.259528:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360351616 00000100:00100000:3.0:1788572542.259579:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4075 00000100:00100000:3.0:1788572542.259590:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35906:x1875453360351616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.259611:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.259628:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 2/0,0 mode: --/CR res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9b4 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.259698:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.259712:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.259732:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572542.259735:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572542.259737:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cdc00, rc 0) 00000100:00100000:3.0:1788572542.259797:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35906:x1875453360351616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 208us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.259804:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4075 00000100:00100000:1.0:1788572542.263279:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.263322:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.263356:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.263425:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360351744 00000100:00100000:2.0:1788572542.263435:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360351744 00000100:00100000:2.0:1788572542.263504:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1071 00000100:00100000:2.0:1788572542.263515:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360351744:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.263742:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360352000 00000100:00100000:3.0:1788572542.263746:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360352000 00000100:00100000:3.0:1788572542.263783:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360352128 00000100:00100000:3.0:1788572542.263787:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360352128 00000100:00100000:3.0:1788572542.263809:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4076 00000100:00100000:3.0:1788572542.263815:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35905:x1875453360352000:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.263832:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360351744:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 320us (566us total) trans 4294967576 rc 0/0 00000100:00100000:2.0:1788572542.263840:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1071 00010000:00010000:3.0:1788572542.263882:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572542.263909:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb162598 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef99f expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.263938:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:3.0:1788572542.263957:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050c00/0x85ace67feb1625bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9a6 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.263966:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572542.263969:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.263979:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4077 00000100:00100000:0.0:1788572542.263989:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35907:x1875453360352128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.263992:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264028:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.264033:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264042:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1625ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9bb expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264044:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264059:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264065:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264072:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264088:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264096:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264103:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264116:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264122:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264128:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264256:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264263:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264270:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264276:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.264281:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264288:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264298:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572542.264301:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.264305:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264312:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb1625e5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264320:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264325:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264331:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264337:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.264340:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264346:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264352:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162608 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264353:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xfe6bf9ea (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.264356:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.264359:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264360:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162608 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.264366:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1625f3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xfe6bf9ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264368:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162608 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.264393:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572542.264398:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000100:00100000:3.0:1788572542.264409:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35905:x1875453360352000:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 594us (1086us total) trans 0 rc -2/-2 00000004:00010000:0.0:1788572542.264414:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162608 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.264416:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4076 00010000:00010000:0.0:1788572542.264423:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264429:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264435:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264440:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.264442:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264448:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264467:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572542.264470:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.264473:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264479:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1625fa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264493:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264499:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264508:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264512:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.264515:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264520:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264527:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627e7451100) 00010000:00010000:0.0:1788572542.264529:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.264532:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264538:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162601 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264566:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1625ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9bb expref: 92 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264578:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162608 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9bb expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.264586:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572542.264588:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572542.264590:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572542.264628:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35907:x1875453360352128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 641us (1270us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.264637:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4077 00000100:00100000:3.0:1788572542.266243:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382351360:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788572542.266305:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572542.266498:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382351360 00000100:00100000:1.0:1788572542.266505:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382351360 00000100:00100000:1.0:1788572542.266560:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:1.0:1788572542.266570:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962800a49050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382351360:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788572542.266591:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 159842304 pending 0 free 3858759680 avail 3853516800 00000100:00100000:1.0:1788572542.266646:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962800a49050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382351360:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.266653:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1788572542.266746:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382351360:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788572542.270640:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.271696:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.271740:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360352256 00000100:00100000:2.0:1788572542.271751:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360352256 00000100:00100000:0.0:1788572542.271782:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360352512 00000100:00100000:0.0:1788572542.271787:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360352512 00000100:00100000:0.0:1788572542.271812:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4078 00000100:00100000:0.0:1788572542.271821:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35905:x1875453360352256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.271841:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.271870:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16260f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9c2 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.271906:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.271919:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.271935:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.272109:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16260f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 94 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.272131:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.272139:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:0.0:1788572542.272142:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572542.272146:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094de00, rc 0) 00000100:00100000:0.0:1788572542.272215:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35905:x1875453360352256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 385us (1574us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.272472:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4079 00000100:00100000:2.0:1788572542.272483:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35907:x1875453360352512:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.272517:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572542.272533:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1625c9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9ad expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272545:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627d0ee2200) 00010000:00010000:2.0:1788572542.272560:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162608 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9bb expref: 92 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272570:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572542.272573:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572542.272590:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272599:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272606:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272619:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272625:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272631:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272887:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272896:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272902:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272908:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.272913:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272921:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272929:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572542.272932:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.272936:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272942:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16261d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272950:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272956:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272961:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272965:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.272967:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272972:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272978:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xf955b15a (ffff9627e7a95f00) 00010000:00010000:2.0:1788572542.272980:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.272984:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.272990:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162624 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xf955b15a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.273064:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35907:x1875453360352512:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 582us (1372us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572542.273071:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4079 00000100:00100000:0.0:1788572542.276179:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4078 00000100:00100000:2.0:1788572542.276875:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.276956:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.277318:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.277516:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360352768 00000100:00100000:1.0:1788572542.277526:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360352768 00000100:00100000:1.0:1788572542.277603:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360352896 00000100:00100000:1.0:1788572542.277607:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360352896 00000100:00100000:3.0:1788572542.277651:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4080 00000100:00100000:3.0:1788572542.277663:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35905:x1875453360352768:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.277759:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.277776:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.277790:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572542.278233:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].58, 88 off12888 00000004:00010000:3.0:1788572542.278345:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000085525022 reply state 00000000bcc08dff transno 4294970367 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.278356:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572542.278451:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35905:x1875453360352768:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 790us (1578us total) trans 4294970367 rc 0/0 00000100:00100000:3.0:1788572542.278471:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4080 00000100:00100000:3.0:1788572542.278493:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4081 00000100:00100000:3.0:1788572542.278501:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35907:x1875453360352896:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.278512:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.278526:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162632 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9c9 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.278551:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.278559:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.278566:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.278634:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162632 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.278656:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 93 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.278664:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.278667:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572542.278671:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc000, rc 0) 00000100:00100000:3.0:1788572542.278696:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35907:x1875453360352896:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 197us (1740us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.278703:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4081 00000100:00100000:0.0:1788572542.278808:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360353152 00000100:00100000:0.0:1788572542.278815:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360353152 00000100:00100000:0.0:1788572542.278855:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 890 00000100:00100000:0.0:1788572542.278863:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35906:x1875453360353152:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.279088:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35906:x1875453360353152:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 226us (1770us total) trans 4294970368 rc 0/0 00000100:00100000:0.0:1788572542.279096:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 890 00010000:00010000:1.0:1788572542.281156:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.281171:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.281178:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.281183:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.281187:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.281195:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.281222:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:1.0:1788572542.281225:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572542.281230:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.281236:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16262b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.282256:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.282323:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.288048:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360353280 00000100:00100000:1.0:1788572542.288063:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360353280 00000100:00100000:1.0:1788572542.288119:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360353536 00000100:00100000:1.0:1788572542.288122:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360353536 00000100:00100000:1.0:1788572542.288145:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4082 00000100:00100000:1.0:1788572542.288152:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35905:x1875453360353280:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.288174:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.288192:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9d0 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.288258:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.288269:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.288283:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:1.0:1788572542.288286:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572542.288289:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:1.0:1788572542.288348:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35905:x1875453360353280:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 196us (6100us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.288353:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4082 00000100:00100000:1.0:1788572542.288371:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4083 00000100:00100000:1.0:1788572542.288375:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35907:x1875453360353536:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.288430:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.288438:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.288444:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.288984:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000040:00080000:1.0:1788572542.289440:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].61, 88 off13080 00000004:00010000:1.0:1788572542.289546:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004b95827d reply state 00000000bf5ce409 transno 4294970369 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.289556:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572542.289681:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35907:x1875453360353536:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1303us (7357us total) trans 4294970369 rc 0/0 00000100:00100000:1.0:1788572542.289689:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4083 00000100:00100000:1.0:1788572542.289711:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360353792 00000100:00100000:1.0:1788572542.289716:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360353792 00000100:00100000:2.0:1788572542.289764:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4084 00000100:00100000:2.0:1788572542.289777:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35906:x1875453360353792:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.289837:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35906:x1875453360353792:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 63us (854us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.289844:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4084 00000100:00100000:3.0:1788572542.295578:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.295608:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.295692:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.295720:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360354048 00000100:00100000:3.0:1788572542.295729:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360354048 00000100:00100000:3.0:1788572542.295776:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360354176 00000100:00100000:3.0:1788572542.295779:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360354176 00000100:00100000:3.0:1788572542.295798:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4085 00000100:00100000:3.0:1788572542.295808:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35907:x1875453360354048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.295831:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.295850:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9d7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.295912:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 98 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.295941:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 98 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.295960:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.295964:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572542.295967:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:3.0:1788572542.296050:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35907:x1875453360354048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 243us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.296058:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4085 00000100:00100000:3.0:1788572542.296135:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4086 00000100:00100000:3.0:1788572542.296143:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35906:x1875453360354176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.296151:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.296160:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162655 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9de expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.296218:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16265c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.296228:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16265c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.296237:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16265c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572542.296267:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572542.296287:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16265c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.296326:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162655 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9de expref: 99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.296341:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16265c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9de expref: 98 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.296348:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572542.296350:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572542.296353:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572542.296379:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35906:x1875453360354176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 237us (687us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.296386:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4086 00010000:00010000:1.0:1788572542.296845:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.296855:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.296861:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.296866:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.296871:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.296878:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.296889:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:1.0:1788572542.296891:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572542.296895:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.296900:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162647 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.297207:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360353920 00000100:00100000:0.0:1788572542.297213:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360353920 00000100:00100000:0.0:1788572542.297261:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1072 00000100:00100000:0.0:1788572542.297270:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360353920:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.297609:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360353920:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 340us (2042us total) trans 4294967577 rc 0/0 00000100:00100000:0.0:1788572542.297617:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1072 00000100:00100000:3.0:1788572542.300719:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.300804:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.300874:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360354432 00000100:00100000:0.0:1788572542.300884:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360354432 00000100:00100000:0.0:1788572542.300945:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360354304 00000100:00100000:0.0:1788572542.300948:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360354304 00000100:00100000:0.0:1788572542.300978:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1073 00000100:00100000:0.0:1788572542.300988:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360354432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.301034:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.301057:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9e5 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572542.301092:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x48:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737d, blocks: 0x1 00010000:00010000:0.0:1788572542.301112:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=890s v=(0 0) 00010000:00010000:0.0:1788572542.301117:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.301128:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.301148:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.301161:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:0.0:1788572542.301216:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360354432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 229us (895us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.301224:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1073 00000100:00100000:0.0:1788572542.301242:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1074 00000100:00100000:0.0:1788572542.301249:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360354304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.301515:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360354304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 265us (710us total) trans 4294967577 rc 0/0 00000100:00100000:0.0:1788572542.301522:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1074 00000100:00100000:1.0:1788572542.303412:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.303573:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360354688 00000100:00100000:2.0:1788572542.303582:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360354688 00000100:00100000:2.0:1788572542.303635:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 891 00000100:00100000:2.0:1788572542.303801:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35905:x1875453360354688:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.304245:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35905:x1875453360354688:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 600us (842us total) trans 4294970370 rc 0/0 00000100:00100000:2.0:1788572542.304254:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 891 00000100:00100000:1.0:1788572542.304508:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.305170:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360354944 00000100:00100000:2.0:1788572542.305178:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360354944 00000100:00100000:2.0:1788572542.305229:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 892 00000100:00100000:2.0:1788572542.305238:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35907:x1875453360354944:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.305659:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35907:x1875453360354944:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 420us (1149us total) trans 4294970371 rc 0/0 00000100:00100000:2.0:1788572542.305667:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 892 00000100:00100000:1.0:1788572542.306434:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.306573:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360355200 00000100:00100000:2.0:1788572542.306579:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360355200 00000100:00100000:2.0:1788572542.306622:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4087 00000100:00100000:2.0:1788572542.306631:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35905:x1875453360355200:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.306806:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35905:x1875453360355200:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 175us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.306814:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4087 00000100:00100000:3.0:1788572542.307671:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.307709:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.307829:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360355456 00000100:00100000:0.0:1788572542.307837:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360355456 00000100:00100000:0.0:1788572542.307880:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360355712 00000100:00100000:0.0:1788572542.307883:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360355712 00000100:00100000:0.0:1788572542.307913:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4088 00000100:00100000:0.0:1788572542.307921:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35906:x1875453360355456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.307992:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35906:x1875453360355456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 71us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.307999:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4088 00000100:00100000:0.0:1788572542.308085:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4089 00000100:00100000:0.0:1788572542.308093:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35907:x1875453360355712:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.308132:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35907:x1875453360355712:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 40us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.308139:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4089 00000100:00100000:0.0:1788572542.309272:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.309441:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360355840 00000100:00100000:3.0:1788572542.309448:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360355840 00000100:00100000:3.0:1788572542.309554:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4090 00000100:00100000:3.0:1788572542.309563:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35905:x1875453360355840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.309579:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.309599:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16266a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9ec expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.309683:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162671 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.309702:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162671 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.309714:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162671 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572542.309750:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572542.309769:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162671 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.309797:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16266a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9ec expref: 101 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.309814:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162671 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9ec expref: 100 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.309821:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:3.0:1788572542.309824:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572542.309829:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572542.309889:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35905:x1875453360355840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 326us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.309896:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4090 00000100:00100000:1.0:1788572542.310898:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.310968:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360355968 00000100:00100000:0.0:1788572542.310974:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360355968 00000100:00100000:0.0:1788572542.311090:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1075 00000100:00100000:0.0:1788572542.311101:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7663c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35906:x1875453360355968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.311115:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.311129:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9f3 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.311159:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.311167:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.311172:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.311178:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.311223:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.311237:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 14 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.311253:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7876a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382351488:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572542.311328:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f180 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.312999:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.313219:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356096 00000100:00100000:0.0:1788572542.313227:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356096 00000100:00100000:0.0:1788572542.313340:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1076 00000100:00100000:0.0:1788572542.316642:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360356096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.316670:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.316702:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6ef9fa expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572542.316733:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x48:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737d, blocks: 0x1 00010000:00010000:0.0:1788572542.316759:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=890s v=(0 0) 00010000:00010000:0.0:1788572542.316764:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.316774:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.316791:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.316806:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572542.316880:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360356096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3531us (3888us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.316889:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1076 00000100:00100000:1.0:1788572542.317789:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.317812:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356480 00000100:00100000:1.0:1788572542.317818:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356480 00000100:00100000:1.0:1788572542.317857:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 281 00000100:00100000:1.0:1788572542.317866:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:2348:x1875453360356480:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572542.317912:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.317938:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162686 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa08 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.317999:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318031:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318043:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318059:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318066:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318072:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318256:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318264:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318270:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318276:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.318281:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318289:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318302:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.318305:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.318310:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.318316:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16268d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.318470:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.318494:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356224 00000100:00100000:2.0:1788572542.318498:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356224 00010000:00010000:2.0:1788572542.318521:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 15 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.318548:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 967 00000100:00100000:2.0:1788572542.318555:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360356224:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.318566:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.318585:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 15 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.318602:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 15 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.318619:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.318622:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.318630:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 15 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.318639:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 15 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.318676:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 15 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.318692:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382351616:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572542.318761:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9900 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.318871:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572542.319193:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319203:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319210:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319216:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.319219:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319226:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319236:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xb00ad294 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572542.319239:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.319242:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319249:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162694 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xb00ad294 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319286:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162686 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa08 expref: 101 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319295:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.319298:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572542.319302:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162686 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa08 expref: 101 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319310:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2badc00, rc -2) 00000100:00100000:1.0:1788572542.319427:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:2348:x1875453360356480:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1561us (1642us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572542.319436:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 281 00000100:00100000:1.0:1788572542.319498:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.319589:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356352 00000100:00100000:1.0:1788572542.319595:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356352 00000100:00100000:1.0:1788572542.319630:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356736 00000100:00100000:1.0:1788572542.319633:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356736 00000100:00100000:1.0:1788572542.319656:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4091 00000100:00100000:1.0:1788572542.319664:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35907:x1875453360356352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.319666:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4092 00000100:00100000:3.0:1788572542.319674:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35905:x1875453360356736:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.319676:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.319689:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16269b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa01 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319733:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1626a2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.319733:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35905:x1875453360356736:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 59us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.319740:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4092 00010000:00010000:1.0:1788572542.319744:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1626a2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319754:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1626a2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.319791:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572542.319810:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1626a2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319835:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16269b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa01 expref: 102 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319849:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1626a2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa01 expref: 101 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.319857:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:1.0:1788572542.319859:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572542.319862:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad600, rc 0) 00000100:00100000:1.0:1788572542.319884:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35907:x1875453360356352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 221us (1013us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.319891:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4091 00010000:00010000:1.0:1788572542.320268:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb162663 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9e5 expref: 15 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.320286:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7876a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382351488:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.320314:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.320326:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:1.0:1788572542.320692:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7663c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35906:x1875453360355968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9591us (9797us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.320701:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1075 00000100:00100000:0.0:1788572542.327192:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382351616:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572542.327223:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.327288:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360356224:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8731us (8829us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.327299:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 967 00000100:00100000:2.0:1788572542.328078:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.328106:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.328177:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356864 00000100:00100000:1.0:1788572542.328185:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356864 00000100:00100000:1.0:1788572542.328228:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360356992 00000100:00100000:1.0:1788572542.328231:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360356992 00000100:00100000:1.0:1788572542.328253:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1077 00000100:00100000:3.0:1788572542.328258:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1078 00000100:00100000:1.0:1788572542.328261:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35905:x1875453360356864:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.328267:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360356992:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.328278:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.328278:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.328294:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa40 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.328294:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa47 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572542.328320:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x49:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737d, blocks: 0x1 00010000:00010000:1.0:1788572542.328323:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.328331:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=890s v=(0 0) 00010000:00010000:3.0:1788572542.328335:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.328340:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.328343:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572542.328343:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.328348:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.328359:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.328370:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc 0) 00010000:00010000:1.0:1788572542.328377:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.328390:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 14 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.328399:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360356992:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 134us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.328404:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382351744:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.328404:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1078 00000100:00100000:1.0:1788572542.328450:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bdaf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.336558:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.336652:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.336747:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360357248 00000100:00100000:0.0:1788572542.336756:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360357248 00000100:00100000:3.0:1788572542.336770:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382351744:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572542.336796:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 15 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.336800:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.336819:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:0.0:1788572542.336843:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 968 00000100:00100000:0.0:1788572542.336853:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360357248:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.336866:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1788572542.336883:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35905:x1875453360356864:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8622us (8809us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.336892:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1077 00010000:00010000:0.0:1788572542.336893:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 15 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.336907:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 14 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.336937:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb16267f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9fa expref: 14 pid: 13272 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.336958:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.336962:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.336970:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.336980:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.337064:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.337080:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7875c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382351872:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572542.337131:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.337356:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360357120 00000100:00100000:1.0:1788572542.337409:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360357120 00002000:00100000:1.0:1788572542.337444:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x48:0x0] (0->2) req@ffff9626c1bcdf80 x1875453360357120/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.337465:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bcdf80 with x1875453360357120 ext(0->2) 00002000:00010000:1.0:1788572542.337470:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1bcdf80 00000100:00100000:1.0:1788572542.337534:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 248 00000100:00100000:1.0:1788572542.337543:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360357120:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.337962:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360357120:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 419us (1413us total) trans 4294967578 rc 0/0 00000100:00100000:1.0:1788572542.337970:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 248 00002000:00100000:1.0:1788572542.337980:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x48:0x0] (0->2) req@ffff9626c1bcdf80 x1875453360357120/t4294967578(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/448 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.337988:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bcdf80 with x1875453360357120 ext(0->2) 00002000:00010000:1.0:1788572542.337992:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1bcdf80 00000100:00100000:2.0:1788572542.338652:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.338723:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360357504 00000100:00100000:1.0:1788572542.338728:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360357504 00000100:00100000:1.0:1788572542.338773:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4093 00000100:00100000:1.0:1788572542.338781:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35907:x1875453360357504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.338932:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35907:x1875453360357504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 151us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.338940:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4093 00000100:00100000:0.0:1788572542.344706:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7875c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382351872:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572542.344734:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.344787:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360357248:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7935us (8133us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.344798:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 968 00000100:00100000:2.0:1788572542.345380:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.345423:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360357632 00000100:00100000:0.0:1788572542.345430:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360357632 00002000:00100000:0.0:1788572542.345483:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x48:0x0] (0->2) req@ffff9626c4921500 x1875453360357632/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.345494:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4921500 with x1875453360357632 ext(0->2) 00002000:00010000:0.0:1788572542.345499:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4921500 00000100:00100000:0.0:1788572542.345528:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 249 00000100:00100000:0.0:1788572542.345537:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360357632:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.345710:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.345798:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360357760 00000100:00100000:1.0:1788572542.345803:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360357760 00000100:00100000:0.0:1788572542.345957:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360357632:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 420us (577us total) trans 4294967578 rc 0/0 00000100:00100000:0.0:1788572542.345965:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 249 00002000:00100000:0.0:1788572542.345972:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x48:0x0] (0->2) req@ffff9626c4921500 x1875453360357632/t4294967578(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/448 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.345981:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4921500 with x1875453360357632 ext(0->2) 00002000:00010000:0.0:1788572542.345985:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4921500 00000100:00100000:1.0:1788572542.346036:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4094 00000100:00100000:1.0:1788572542.346045:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35906:x1875453360357760:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.355953:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.355991:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.356026:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358016 00000100:00100000:1.0:1788572542.356035:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358016 00000100:00100000:1.0:1788572542.356076:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1079 00000100:00100000:1.0:1788572542.356083:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35907:x1875453360358016:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.356098:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.356112:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa4e expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.356138:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.356144:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.356148:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.356152:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.356176:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.356186:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.356197:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2735c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382352000:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.356265:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bdaf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.356281:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360357888 00000100:00100000:1.0:1788572542.356285:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360357888 00000100:00100000:1.0:1788572542.356305:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4095 00000100:00100000:1.0:1788572542.356309:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35905:x1875453360357888:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.358226:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35905:x1875453360357888:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 1905us (2273us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.358239:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4095 00000100:00100000:0.0:1788572542.365046:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.365437:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358144 00000100:00100000:3.0:1788572542.365449:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358144 00000100:00100000:3.0:1788572542.365557:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1080 00000100:00100000:3.0:1788572542.365568:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360358144:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.373779:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.373841:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2735c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382352000:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.373880:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 17 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.373906:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:1.0:1788572542.373970:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35907:x1875453360358016:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 17885us (17978us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.373981:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1079 00000100:00100000:2.0:1788572542.374785:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358272 00000100:00100000:2.0:1788572542.374796:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358272 00010000:00010000:2.0:1788572542.374836:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.374873:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 969 00000100:00100000:2.0:1788572542.374883:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360358272:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.374898:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.374921:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.374946:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.374970:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70070ee/0x85ace67feb1626b0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa47 expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.374984:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.374988:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.374996:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 16 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.375038:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 16 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.375078:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 16 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.375097:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4922a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382352128:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572542.375157:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9900 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.380017:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4922a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382352128:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572542.380066:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.380117:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360358272:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5235us (6347us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.380129:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 969 00000100:00100000:2.0:1788572542.380192:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.385743:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358400 00000100:00100000:1.0:1788572542.385764:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358400 00002000:00100000:1.0:1788572542.385821:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x49:0x0] (0->2) req@ffff9626c4922a00 x1875453360358400/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.385833:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4922a00 with x1875453360358400 ext(0->2) 00002000:00010000:1.0:1788572542.385839:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4922a00 00000100:00100000:1.0:1788572542.385865:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 250 00000100:00100000:1.0:1788572542.385875:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360358400:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.386305:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360358400:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 429us (6111us total) trans 4294967579 rc 0/0 00000100:00100000:0.0:1788572542.388850:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.388920:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358528 00000100:00100000:2.0:1788572542.388933:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358528 00000100:00100000:2.0:1788572542.388983:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4096 00000100:00100000:2.0:1788572542.388992:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35907:x1875453360358528:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.389115:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35907:x1875453360358528:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 124us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.389122:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4096 00000100:00100000:2.0:1788572542.393977:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.394314:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 250 00002000:00100000:1.0:1788572542.394334:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x49:0x0] (0->2) req@ffff9626c4922a00 x1875453360358400/t4294967579(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/448 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.394346:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4922a00 with x1875453360358400 ext(0->2) 00002000:00010000:1.0:1788572542.394353:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4922a00 00000100:00100000:0.0:1788572542.396292:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358656 00000100:00100000:0.0:1788572542.396307:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358656 00000100:00100000:0.0:1788572542.398824:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1081 00000100:00100000:0.0:1788572542.398840:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360358656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1788572542.402752:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970368 is committed 00000100:00100000:1.0:1788572542.402763:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35906:x1875453360357760:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 56714us (57113us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.402774:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4094 00000001:00080000:2.0:1788572542.402777:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970330 is committed 00000001:00080000:2.0:1788572542.402786:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970369 is committed 00000001:00080000:2.0:1788572542.402798:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970363 is committed 00000020:00080000:2.0:1788572542.402817:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c3251480 refcount 2 committed 0 result 0 batchid 108 00000020:00080000:2.0:1788572542.402822:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aeb00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c11cac00 00000020:00080000:2.0:1788572542.402826:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627d4b18400 00000020:00080000:2.0:1788572542.402847:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d12b0a80 refcount 2 committed 0 result 0 batchid 106 00000020:00080000:2.0:1788572542.402851:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aed00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d12b00c0 00000020:00080000:2.0:1788572542.402853:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627dff73800 00000001:00080000:2.0:1788572542.402870:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970371 is committed 00000020:00080000:2.0:1788572542.402887:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c99d8240 refcount 2 committed 0 result 0 batchid 107 00000020:00080000:2.0:1788572542.402890:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc900 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c99d8a80 00000020:00080000:2.0:1788572542.402893:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcce00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c375e400 00000020:00080000:2.0:1788572542.402932:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 105 batchid updated 00000020:00080000:1.0:1788572542.403031:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 108 committed batchid 105 00000020:00080000:1.0:1788572542.403287:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 105 00000100:00100000:2.0:1788572542.406843:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.406919:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360358784 00000100:00100000:1.0:1788572542.406930:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360358784 00000100:00100000:1.0:1788572542.407032:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1082 00000100:00100000:1.0:1788572542.407044:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360358784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1788572542.422956:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967578 is committed 00000001:00080000:1.0:1788572542.422980:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967575 is committed 00000001:00080000:1.0:1788572542.422987:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967576 is committed 00000100:00100000:3.0:1788572542.423284:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360358144:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 57711us (58243us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.423297:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1080 00000100:00100000:0.0:1788572542.427628:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.427734:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360359040 00000100:00100000:2.0:1788572542.427746:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360359040 00000100:00100000:2.0:1788572542.427818:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4097 00000100:00100000:2.0:1788572542.427829:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35905:x1875453360359040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.427892:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.427921:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162671 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9ec expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.427940:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:2.0:1788572542.427945:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572542.427977:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.427991:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.428034:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572542.428189:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000448d9c4b reply state 00000000e737bcc9 transno 4294970372 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.428200:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572542.428302:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35905:x1875453360359040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 474us (683us total) trans 4294970372 rc 0/0 00000100:00100000:2.0:1788572542.428311:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4097 00010000:00010000:2.0:1788572542.430779:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.430795:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.430803:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.430809:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.430813:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.430821:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.430839:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:2.0:1788572542.430842:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.430862:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 2/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.430869:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1626be lrc: 0/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.431709:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.431776:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360359168 00000100:00100000:2.0:1788572542.431786:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360359168 00000100:00100000:2.0:1788572542.431832:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1083 00000100:00100000:2.0:1788572542.431842:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360359168:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572542.432093:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x48:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572542.432097:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x48:0x0] updating lvb mtime from disk: 0 -> 1788572520 00002000:00010000:2.0:1788572542.432100:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x48:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:2.0:1788572542.432102:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x48:0x0] updating lvb ctime from disk: 1788572541 -> 1788572524 00002000:00010000:2.0:1788572542.432105:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x48:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572542.432166:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360359168:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 324us (463us total) trans 4294967579 rc 0/0 00000100:00100000:2.0:1788572542.432173:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1083 00000100:00100000:0.0:1788572542.439962:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.440046:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360359424 00000100:00100000:1.0:1788572542.440057:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360359424 00000100:00100000:1.0:1788572542.440126:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4098 00000100:00100000:1.0:1788572542.440136:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35905:x1875453360359424:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.440327:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.440413:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.440424:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.440475:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.440489:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 65 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.440504:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2735c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382352256:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.440631:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.450469:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.450532:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360359552 00000100:00100000:0.0:1788572542.450541:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360359552 00000100:00100000:1.0:1788572542.450543:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2735c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382352256:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.450569:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450578:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.450651:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 970 00000100:00100000:0.0:1788572542.450662:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360359552:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.450679:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.450686:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450700:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450719:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162616 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x82:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c2 expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450733:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:0.0:1788572542.450736:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450743:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450748:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.450768:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572542.450793:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.450800:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.450824:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360359552:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 164us (376us total) trans 0 rc 0/0 00010000:00010000:1.0:1788572542.450828:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.450832:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 970 00010000:00010000:1.0:1788572542.450836:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.450843:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.450850:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.450855:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.450868:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.450878:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:1.0:1788572542.450881:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572542.450885:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.450891:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1626c5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.450945:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35905:x1875453360359424:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10810us (10993us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572542.450953:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4098 00000001:00080000:3.0:1788572542.456969:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967577 is committed 00000001:00080000:3.0:1788572542.456990:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967579 is committed 00000001:00080000:3.0:1788572542.457033:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967575 is committed 00000100:00100000:3.0:1788572542.457149:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360358784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 50101us (50308us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.457160:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1082 00000100:00100000:0.0:1788572542.457809:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360358656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 58970us (63838us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.457818:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1081 00000100:00100000:3.0:1788572542.458615:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.458707:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360359680 00000100:00100000:2.0:1788572542.458717:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360359680 00000100:00100000:2.0:1788572542.458808:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1084 00000100:00100000:2.0:1788572542.458819:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360359680:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.458867:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/1,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.458891:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 11 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.458901:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.458907:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572542.458911:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 4/1,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.458962:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 11 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.458978:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 11 pid: 20911 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.458993:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382352384:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.459099:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.460507:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.460568:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360359936 00000100:00100000:0.0:1788572542.460576:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360359936 00000100:00100000:0.0:1788572542.460668:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4099 00000100:00100000:0.0:1788572542.460678:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35906:x1875453360359936:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.460733:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.460759:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16265c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9de expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.460779:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572542.460784:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572542.460815:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.460827:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.460837:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572542.461045:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002d755d1c reply state 00000000eee29698 transno 4294970373 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.461054:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572542.461138:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35906:x1875453360359936:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 461us (636us total) trans 4294970373 rc 0/0 00000100:00100000:0.0:1788572542.461146:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4099 00000100:00100000:1.0:1788572542.464224:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572542.464402:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.464414:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.464421:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.464427:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.464432:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.464440:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.464470:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.464473:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.464478:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.464490:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1626d3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.464582:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360360192 00000100:00100000:3.0:1788572542.464592:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360360192 00000100:00100000:3.0:1788572542.464647:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4100 00000100:00100000:3.0:1788572542.464658:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35907:x1875453360360192:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.464714:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572542.464732:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1626a2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa01 expref: 64 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.464749:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.464753:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572542.464781:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.464793:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.464803:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572542.464987:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000050258650 reply state 00000000b2e96bfa transno 4294970374 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.464996:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572542.465134:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35907:x1875453360360192:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 477us (927us total) trans 4294970374 rc 0/0 00000100:00100000:3.0:1788572542.465141:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4100 00000100:00100000:3.0:1788572542.467414:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.467526:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360360320 00000100:00100000:2.0:1788572542.467536:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360360320 00000100:00100000:2.0:1788572542.467588:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1085 00000100:00100000:2.0:1788572542.467597:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360360320:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572542.467873:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x48:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572542.467877:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x48:0x0] updating lvb mtime from disk: 0 -> 1788572519 00002000:00010000:2.0:1788572542.467879:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x48:0x0] updating lvb atime from disk: 0 -> 1788572519 00002000:00010000:2.0:1788572542.467881:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x48:0x0] updating lvb ctime from disk: 1788572541 -> 1788572525 00002000:00010000:2.0:1788572542.467883:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x48:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572542.468090:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360360320:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 492us (683us total) trans 4294967580 rc 0/0 00000100:00100000:2.0:1788572542.468099:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1085 00010000:00010000:1.0:1788572542.468181:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.468191:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.468200:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.468330:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.468336:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.468347:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.468363:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:1.0:1788572542.468367:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.468371:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 2/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.468378:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1626da lrc: 0/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.477052:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.477102:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.477162:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382352384:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.477191:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/1,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.477286:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360360576 00000100:00100000:3.0:1788572542.477294:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360360576 00000100:00100000:3.0:1788572542.477363:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1086 00000100:00100000:3.0:1788572542.477373:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360360576:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572542.477604:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x49:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572542.477607:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x49:0x0] updating lvb mtime from disk: 0 -> 1788572520 00002000:00010000:3.0:1788572542.477609:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x49:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:3.0:1788572542.477611:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x49:0x0] updating lvb ctime from disk: 1788572541 -> 1788572524 00002000:00010000:3.0:1788572542.477614:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x49:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572542.477673:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360360576:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 301us (570us total) trans 4294967581 rc 0/0 00000100:00100000:3.0:1788572542.477681:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1086 00000100:00100000:1.0:1788572542.480567:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360360448 00000100:00100000:1.0:1788572542.480581:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360360448 00010000:00010000:1.0:1788572542.480623:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 12 pid: 20911 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.480774:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 971 00000100:00100000:1.0:1788572542.480785:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360360448:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.480800:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.480825:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 12 pid: 20911 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.480838:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 12 pid: 20911 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.480864:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb1626a9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa40 expref: 12 pid: 20911 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.480888:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.480892:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.480900:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.480911:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.480999:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572542.481362:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360360448:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 577us (4319us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.481372:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 971 00010000:00010000:2.0:1788572542.481912:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.481924:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.481958:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.481968:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.481976:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.481982:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572542.481986:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.481996:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.482237:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.482247:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb1626cc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.488900:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.489581:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.489640:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360360832 00000100:00100000:3.0:1788572542.489652:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360360832 00000100:00100000:3.0:1788572542.489710:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361088 00000100:00100000:3.0:1788572542.489714:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361088 00000100:00100000:3.0:1788572542.489747:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4101 00000100:00100000:3.0:1788572542.489757:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35906:x1875453360360832:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.489829:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.489847:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.489857:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.489888:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.489902:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 65 pid: 5837 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.489916:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382352512:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.490442:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.490476:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4102 00000100:00100000:3.0:1788572542.490486:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35907:x1875453360361088:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.490516:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.490526:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.490534:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.490551:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.490560:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 65 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.490570:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382352640:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.490590:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.492874:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360359680:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 34044us (34248us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.492893:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1084 00000100:00100000:0.0:1788572542.494266:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382352640:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572542.494297:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.494559:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382352512:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.494575:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.494782:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.494857:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.495993:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.496085:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361600 00000100:00100000:3.0:1788572542.496094:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361600 00000100:00100000:3.0:1788572542.496152:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4103 00000100:00100000:3.0:1788572542.496163:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35905:x1875453360361600:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.496247:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 3/0,1 mode: --/PW res: [0x200000401:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496265:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496277:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496556:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496568:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496576:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496613:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.496628:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 67 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.496778:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382352768:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.496856:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f540 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.497612:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361216 00000100:00100000:2.0:1788572542.497620:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361216 00000100:00100000:1.0:1788572542.497852:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361344 00000100:00100000:1.0:1788572542.497857:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361344 00010000:00010000:1.0:1788572542.497892:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 69 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.497932:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 972 00000100:00100000:1.0:1788572542.497942:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e53450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2349:x1875453360361344:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.497954:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.497961:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 69 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.497972:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 69 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.497990:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162639 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x83:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9c9 expref: 69 pid: 5838 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.498028:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:1.0:1788572542.498033:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.498043:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.498050:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.498079:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572542.498217:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498226:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498252:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498259:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498266:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498272:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.498277:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498284:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498294:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:2.0:1788572542.498297:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572542.498300:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.498307:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1626e8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498485:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 68 pid: 5837 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.498508:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 973 00000100:00100000:3.0:1788572542.498515:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360361216:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572542.498523:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572542.498528:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 68 pid: 5837 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498539:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 68 pid: 5837 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498551:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16258a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x81:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef998 expref: 68 pid: 5837 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498561:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572542.498565:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498572:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498578:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.498597:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572542.498757:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e53450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360361344:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 814us (3897us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.498765:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 972 00000100:00100000:2.0:1788572542.498814:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35907:x1875453360361088:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8330us (9235us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572542.498822:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4102 00000100:00100000:3.0:1788572542.498867:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360361216:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 352us (4086us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.498874:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 973 00010000:00010000:2.0:1788572542.499104:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499112:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499130:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499136:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499142:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499146:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.499149:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499155:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499163:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.499166:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572542.499168:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.499177:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb1626e1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.499212:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35906:x1875453360360832:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 9457us (10325us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572542.499220:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4101 00000100:00100000:0.0:1788572542.504414:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.504512:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.504624:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382352768:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.504668:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.504805:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.504909:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361984 00000100:00100000:2.0:1788572542.504917:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361984 00010000:00010000:2.0:1788572542.504963:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 64 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.504998:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 974 00000100:00100000:2.0:1788572542.505026:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2348:x1875453360361984:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.505040:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.505046:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 64 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.505059:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 64 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.505080:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb162640 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d0 expref: 64 pid: 9040 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.505097:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:2.0:1788572542.505102:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.505116:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.505123:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.505144:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.505190:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2348:x1875453360361984:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 185us (386us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.505197:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 974 00000100:00100000:3.0:1788572542.505406:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361728 00000100:00100000:3.0:1788572542.505410:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361728 00000100:00100000:3.0:1788572542.505484:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360361856 00000100:00100000:3.0:1788572542.505487:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360361856 00000100:00100000:1.0:1788572542.505546:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1087 00000100:00100000:1.0:1788572542.505555:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360361728:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.505588:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.505606:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.505613:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.505617:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.505621:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 4/1,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.505661:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.505674:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 14 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.505688:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2735500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382352896:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.505734:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb540 going to sleep for 11 seconds 00010000:00010000:1.0:1788572542.505750:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.505756:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572542.505952:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970375 00000001:00080000:1.0:1788572542.506127:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970376 00000001:00080000:1.0:1788572542.506172:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970377 00000100:00100000:1.0:1788572542.506266:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2736a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382353024:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572542.506320:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572542.506541:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57cc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.506756:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1088 00000100:00100000:2.0:1788572542.506768:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360361856:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.506792:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.506806:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.506813:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.506817:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572542.506821:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.506843:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.506859:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 14 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.506869:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382353152:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.506907:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.507062:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382353024 00000100:00100000:2.0:1788572542.507068:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382353024 00000100:00100000:2.0:1788572542.507103:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:2.0:1788572542.507110:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382353024:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572542.507176:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382353024:12345-0@lo:1000:mdt00_004.0 Request processed in 66us (855us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.507183:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:0.0:1788572542.507256:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967576 gen 2 00000004:00080000:0.0:1788572542.507263:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfae7100, transno 4294967570 00000004:00080000:0.0:1788572542.507281:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfae4a80, transno 4294967571 00000004:00080000:0.0:1788572542.507284:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627d131d180, transno 4294967572 00000004:00080000:0.0:1788572542.507288:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967575 committed transno 4294967576 00000004:00080000:0.0:1788572542.507305:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967576 committed transno 4294967576 00000100:00100000:0.0:1788572542.507325:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2736a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382353024:0@lo:1000:mdt00_004.0 00000001:00080000:0.0:1788572542.507775:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970378 00000100:00100000:0.0:1788572542.507836:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382353280:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572542.507872:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572542.507896:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.507916:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382353280 00000100:00100000:0.0:1788572542.507919:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382353280 00000100:00100000:0.0:1788572542.507960:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1788572542.507967:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382353280:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572542.508150:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382353280:12345-0@lo:1000:mdt00_004.0 Request processed in 182us (276us total) trans 4294967580 rc 0/0 00000100:00100000:0.0:1788572542.508158:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1788572542.508195:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382353280:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1788572542.508224:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508232:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508239:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508245:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.508251:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508261:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508276:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:0.0:1788572542.508280:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572542.508285:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508291:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb1626f6 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x82:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508326:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508333:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508339:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508344:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d4e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.508347:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508353:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508361:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627ffc16c00) 00010000:00010000:0.0:1788572542.508363:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572542.508366:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508371:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d4e00/0x85ace67feb15f1a4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508391:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508402:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508410:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508418:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508423:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508430:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508435:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.508437:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508443:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508450:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627ffc16c00) 00010000:00010000:0.0:1788572542.508452:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.508465:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 2/0,0 mode: --/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508471:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16270b lrc: 0/0,0 mode: --/EX res: [0x200000401:0x84:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508483:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508489:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508494:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508499:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.508502:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508511:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508519:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627ffc16c00) 00010000:00010000:0.0:1788572542.508521:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572542.508524:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 2/0,0 mode: --/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.508529:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1626ef lrc: 0/0,0 mode: --/PW res: [0x200000401:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572542.509065:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 2 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:0.0:1788572542.509179:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35905:x1875453360361600:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 13015us (13186us total) trans 4294970379 rc 0/0 00000100:00100000:0.0:1788572542.509188:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4103 00000020:00080000:0.0:1788572542.509192:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572542.516431:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2735500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382352896:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.516474:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.516552:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.516625:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360362112 00000100:00100000:0.0:1788572542.516633:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360362112 00010000:00010000:0.0:1788572542.516685:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 15 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.516732:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 975 00000100:00100000:0.0:1788572542.516742:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360362112:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.516760:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.516788:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 15 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.516804:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 15 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.516831:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb1626b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa4e expref: 15 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.516850:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.516855:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.516862:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.516872:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.516914:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.516977:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360362112:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 236us (426us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.516985:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 975 00010000:00010000:3.0:1788572542.517376:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517392:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517419:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517428:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517436:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517444:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572542.517448:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517468:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517481:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.517489:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1626fd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.517558:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360361728:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 12004us (13155us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.517566:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1087 00000100:00100000:2.0:1788572542.518420:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382353152:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.518444:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.531253:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.531361:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.531406:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.531489:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360362752 00000100:00100000:0.0:1788572542.531495:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360362496 00000100:00100000:0.0:1788572542.531501:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360362496 00000100:00100000:1.0:1788572542.531501:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360362752 00000100:00100000:0.0:1788572542.531554:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 893 00000100:00100000:1.0:1788572542.531554:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4104 00000100:00100000:0.0:1788572542.531564:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35905:x1875453360362496:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.531564:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35907:x1875453360362752:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000004:00080000:0.0:1788572542.532519:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x82:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572542.532622:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970380 00010000:00010000:0.0:1788572542.532670:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.532688:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.532704:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.532712:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.532718:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.532723:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.532742:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572542.532845:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35905:x1875453360362496:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1281us (1482us total) trans 4294970381 rc 0/0 00000100:00100000:0.0:1788572542.532853:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 893 00000020:00080000:0.0:1788572542.532858:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572542.533162:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572542.533170:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.533177:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572542.533180:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.533189:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.533201:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x82:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:3.0:1788572542.533204:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572542.533209:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.533216:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162712 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x82:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.533236:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572542.533238:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572542.533240:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572542.533986:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.534024:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.534035:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.534524:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360362240 00000100:00100000:2.0:1788572542.534533:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360362240 00010000:00010000:2.0:1788572542.535758:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 13 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.535845:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 976 00000100:00100000:2.0:1788572542.535867:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360362240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.535885:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.535913:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 13 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.535938:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 13 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.535959:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008806a641/0x85ace67feb162678 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x48:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9f3 expref: 13 pid: 10416 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.535977:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572542.535982:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572542.535988:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.535998:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.536050:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.536119:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360362240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 255us (4876us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.536127:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 976 00010000:00010000:0.0:1788572542.538122:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538141:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.538149:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538153:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.538158:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538187:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538195:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538204:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538211:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572542.538216:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538225:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538243:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.538252:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb162704 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.538269:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.538288:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 69 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.538305:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7876680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382353408:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.538357:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360361856:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 31590us (33843us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.538366:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1088 00000100:00100000:0.0:1788572542.538598:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.542525:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.542687:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7876680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382353408:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572542.542714:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.542810:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360363008 00000100:00100000:2.0:1788572542.542817:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360363008 00000100:00100000:2.0:1788572542.542878:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4105 00000100:00100000:2.0:1788572542.542887:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35906:x1875453360363008:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.542987:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.542999:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.543031:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.543228:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.543239:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 71 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.543247:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 4/0,1 mode: --/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.543281:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 71 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.543291:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 71 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.543301:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382353536:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.543384:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.544070:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.544234:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360363136 00000100:00100000:2.0:1788572542.544242:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360363136 00010000:00010000:2.0:1788572542.544268:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 72 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.544308:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 977 00000100:00100000:2.0:1788572542.544316:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2349:x1875453360363136:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.544330:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.544338:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 72 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.544349:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 72 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.544369:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16264e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9d7 expref: 72 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.544383:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:2.0:1788572542.544388:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.544396:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.544403:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.544431:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.544759:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2349:x1875453360363136:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 443us (691us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.544768:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 977 00010000:00010000:3.0:1788572542.546061:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.546075:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572542.546290:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970382 00000001:00080000:3.0:1788572542.546493:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970384 00000001:00080000:3.0:1788572542.546546:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970385 00000100:00100000:3.0:1788572542.546638:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382353664:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572542.546713:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572542.546757:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.546926:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382353664 00000100:00100000:2.0:1788572542.546934:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382353664 00000100:00100000:2.0:1788572542.546990:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:2.0:1788572542.546998:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382353664:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1788572542.547095:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382353664:12345-0@lo:1000:mdt00_001.0 Request processed in 98us (382us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.547102:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1788572542.547709:0:5838:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967575 gen 2 00000004:00080000:3.0:1788572542.547716:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967575 committed transno 4294967575 00000100:00100000:3.0:1788572542.547739:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382353664:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1788572542.548161:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970386 00000100:00100000:3.0:1788572542.548240:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382353792:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572542.548285:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572542.548309:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5e00300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.548385:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382353792 00000100:00100000:0.0:1788572542.548392:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382353792 00000100:00100000:0.0:1788572542.548433:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:0.0:1788572542.548442:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382353792:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572542.548635:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382353792:12345-0@lo:1000:mdt00_001.0 Request processed in 193us (348us total) trans 4294967582 rc 0/0 00000100:00100000:0.0:1788572542.548643:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1788572542.548857:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382353792:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1788572542.548893:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548902:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548910:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548916:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.548920:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548928:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548945:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.548948:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572542.548953:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548959:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162720 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x83:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.548997:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549169:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549176:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549183:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb5d5800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.549185:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549191:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549200:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627ffc16900) 00010000:00010000:3.0:1788572542.549203:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572542.549206:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549212:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb5d5800/0x85ace67feb15f2bc lrc: 0/0,0 mode: --/EX res: [0x200000401:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549237:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549250:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549260:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549267:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549273:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549278:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549284:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.549287:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549293:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549300:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627ffc16900) 00010000:00010000:3.0:1788572542.549302:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.549305:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549311:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162735 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x85:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549322:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549327:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549334:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549339:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.549341:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549346:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549358:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627ffc16900) 00010000:00010000:3.0:1788572542.549360:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572542.549363:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.549369:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162719 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.549516:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35907:x1875453360362752:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 17951us (18106us total) trans 4294970387 rc 0/0 00000100:00100000:3.0:1788572542.549526:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4104 00000020:00080000:3.0:1788572542.549530:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572542.552345:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.552599:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360363264 00000100:00100000:0.0:1788572542.552609:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360363264 00010000:00010000:0.0:1788572542.552661:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 74 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.552703:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 978 00000100:00100000:0.0:1788572542.552713:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360363264:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.552728:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.552734:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 74 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.552747:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 74 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.552776:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572542.552782:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.552791:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.552799:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.552819:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.552872:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360363264:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 161us (546us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.552880:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 978 00010000:00010000:2.0:1788572542.553830:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1625de lrc: 0/0,0 mode: --/CR res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef9b4 expref: 73 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.553850:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382353536:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.553878:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.553888:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.553894:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572542.554425:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970383 00000001:00080000:2.0:1788572542.554590:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970388 00000001:00080000:2.0:1788572542.554630:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970389 00000100:00100000:2.0:1788572542.554784:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382353920:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572542.554848:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.554922:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.555111:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382353920 00000100:00100000:0.0:1788572542.555119:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382353920 00000100:00100000:0.0:1788572542.555224:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:0.0:1788572542.555233:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382353920:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572542.555321:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382353920:12345-0@lo:1000:mdt00_000.0 Request processed in 89us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.555328:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1788572542.555396:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382353920:0@lo:1000:mdt00_000.0 00000001:00080000:2.0:1788572542.555761:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970390 00000100:00100000:2.0:1788572542.555818:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382354048:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572542.555851:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.556474:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572542.556565:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382354048 00000100:00100000:1.0:1788572542.556572:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382354048 00000100:00100000:1.0:1788572542.556610:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1788572542.556617:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382354048:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572542.557478:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:5837:x1875453382354048:12345-0@lo:1000:mdt00_000.0 Request processed in 856us (1621us total) trans 4294967583 rc 0/0 00000100:00100000:1.0:1788572542.557488:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:0.0:1788572542.557617:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.557736:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360363520 00000100:00100000:1.0:1788572542.557743:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360363520 00000100:00100000:1.0:1788572542.557790:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 894 00000100:00100000:1.0:1788572542.557799:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35907:x1875453360363520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.558436:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4923100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382354048:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1788572542.558480:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558491:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558500:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558507:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.558512:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558520:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558536:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.558540:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572542.558545:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558551:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16272e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x81:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558584:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558591:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558597:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558602:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.558605:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558611:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558619:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d1866600) 00010000:00010000:2.0:1788572542.558622:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572542.558624:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.558630:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb15f23e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572542.558749:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x83:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572542.558795:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970392 00010000:00010000:1.0:1788572542.558838:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.558853:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.558864:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.558873:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.558879:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.558884:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.559142:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572542.559255:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35907:x1875453360363520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1455us (1638us total) trans 4294970393 rc 0/0 00000100:00100000:1.0:1788572542.559264:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 894 00000020:00080000:1.0:1788572542.559269:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572542.559894:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559907:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559916:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559934:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559941:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559948:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559954:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.559958:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559966:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559976:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d1866600) 00010000:00010000:2.0:1788572542.559978:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.559981:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559987:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162743 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.559997:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.560025:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.560031:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.560036:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.560039:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.560047:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.560055:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d1866600) 00010000:00010000:2.0:1788572542.560056:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572542.560059:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.560064:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162727 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.560322:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35906:x1875453360363008:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 17434us (17807us total) trans 4294970391 rc 0/0 00000100:00100000:2.0:1788572542.560333:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4105 00000020:00080000:2.0:1788572542.560429:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1788572542.560912:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572542.560917:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572542.560954:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 136 count 101 00000040:00080000:1.0:1788572542.560960:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:1.0:1788572542.561115:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2735500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382354176:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572542.561169:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9600 going to sleep for 11 seconds 00010000:00010000:0.0:1788572542.562119:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572542.562130:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.562139:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572542.562143:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.562153:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.562169:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x83:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:0.0:1788572542.562172:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572542.562177:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.562182:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16273c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x83:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.562200:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572542.562202:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572542.562204:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572542.569104:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.569350:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360363776 00000100:00100000:0.0:1788572542.569362:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360363776 00000100:00100000:0.0:1788572542.569431:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 895 00000100:00100000:0.0:1788572542.569442:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35906:x1875453360363776:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572542.569929:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x81:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572542.569999:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970394 00010000:00010000:0.0:1788572542.570084:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.570102:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.570115:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.570122:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.570130:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.570136:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.570151:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572542.570261:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35906:x1875453360363776:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 819us (1165us total) trans 4294970395 rc 0/0 00000100:00100000:0.0:1788572542.570269:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 895 00000020:00080000:0.0:1788572542.570275:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572542.570648:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572542.570657:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.570664:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572542.570669:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.570677:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.570690:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x81:0x0].0x0 (ffff9627fb1e7300) 00010000:00010000:2.0:1788572542.570693:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572542.570698:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 1/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.570707:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb16274a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x81:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.570731:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572542.570733:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572542.570735:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572542.572812:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2735500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382354176:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572542.573438:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.573481:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.573728:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.573903:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360363904 00000100:00100000:2.0:1788572542.573911:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360363904 00000100:00100000:2.0:1788572542.573976:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360364032 00000100:00100000:2.0:1788572542.573979:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360364032 00000100:00100000:2.0:1788572542.573993:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360364160 00000100:00100000:2.0:1788572542.573996:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360364160 00000100:00100000:2.0:1788572542.574038:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4106 00000100:00100000:2.0:1788572542.574048:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35912:x1875453360363904:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572542.574084:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x88:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572542.574193:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35912:x1875453360363904:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 147us (756us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.574201:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4106 00000100:00100000:2.0:1788572542.574234:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4107 00000100:00100000:2.0:1788572542.574242:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35910:x1875453360364032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572542.574257:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x89:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572542.574300:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35910:x1875453360364032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 59us (817us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.574307:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4107 00000100:00100000:2.0:1788572542.574320:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4108 00000100:00100000:2.0:1788572542.574326:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35911:x1875453360364160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572542.574352:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x83:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572542.574396:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35911:x1875453360364160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 70us (669us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.574403:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4108 00000100:00100000:3.0:1788572542.579639:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.579718:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.579876:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360364288 00000100:00100000:2.0:1788572542.579892:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360364288 00000100:00100000:2.0:1788572542.579978:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360364416 00000100:00100000:2.0:1788572542.579982:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360364416 00000100:00100000:2.0:1788572542.580033:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4109 00000100:00100000:2.0:1788572542.580044:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35912:x1875453360364288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.580123:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580141:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580153:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580167:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580173:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580180:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580214:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580227:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580251:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.580292:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x88:0x0]: returning size 3 00000004:00200000:2.0:1788572542.580297:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572542.580313:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580325:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580346:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580353:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580360:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.580365:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580373:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580385:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572542.580388:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.580393:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580403:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb162751 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580416:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580422:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580427:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580432:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.580434:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580440:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580447:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xe3f2b718 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.580450:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.580464:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580470:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb162758 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xe3f2b718 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580485:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580491:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580498:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580503:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.580506:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580511:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580519:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:2.0:1788572542.580522:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.580525:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580531:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb16275f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.580608:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35912:x1875453360364288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 564us (979us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.580617:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4109 00000100:00100000:2.0:1788572542.580640:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4110 00000100:00100000:2.0:1788572542.580649:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35910:x1875453360364416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.580682:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580691:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580703:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580716:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580722:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580728:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580745:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580751:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580756:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572542.580777:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x89:0x0]: returning size 3 00000004:00200000:2.0:1788572542.580781:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572542.580787:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580793:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580798:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580808:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580813:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.580816:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580822:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580829:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572542.580831:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.580834:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580839:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162766 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580847:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580853:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580859:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580863:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.580866:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580875:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580882:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x86fbd933 (ffff9627e7a95200) 00010000:00010000:2.0:1788572542.580884:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.580887:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580893:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb16276d lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x86fbd933 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580905:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580911:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580916:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580920:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.580923:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580929:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580936:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:2.0:1788572542.580938:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.580941:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.580952:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162774 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.580985:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35910:x1875453360364416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 338us (1267us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.580992:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4110 00000100:00100000:3.0:1788572542.582200:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.582663:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360364544 00000100:00100000:1.0:1788572542.582674:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360364544 00000100:00100000:1.0:1788572542.582734:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4111 00000100:00100000:1.0:1788572542.582744:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35911:x1875453360364544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.582810:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.582825:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.582836:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583169:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583179:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583187:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583227:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583235:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583248:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.583291:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x83:0x0]: returning size 3 00000004:00200000:1.0:1788572542.583297:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572542.583310:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583322:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583426:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583436:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583443:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.583447:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583465:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583477:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572542.583480:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.583485:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583491:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16277b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583507:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583513:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583519:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583524:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.583527:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583533:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583541:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x35d505c3 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572542.583543:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.583546:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583551:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162782 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x35d505c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583567:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583573:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583579:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583585:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.583588:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583594:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583601:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:1.0:1788572542.583604:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.583607:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.583613:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162789 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.583762:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35911:x1875453360364544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1014us (1564us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.583771:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4111 00000100:00100000:3.0:1788572542.586867:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.587049:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.587735:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360364800 00000100:00100000:2.0:1788572542.587746:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360364800 00000100:00100000:0.0:1788572542.587804:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360365056 00000100:00100000:0.0:1788572542.587808:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360365056 00000100:00100000:0.0:1788572542.587844:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4112 00000100:00100000:0.0:1788572542.587853:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35910:x1875453360364800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.587873:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.587896:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162790 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa55 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588031:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588046:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588057:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.588069:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x89:0x0]: returning size 3 00010000:00010000:0.0:1788572542.588171:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588178:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588184:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588189:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.588193:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588199:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588227:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572542.588231:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572542.588236:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588242:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162797 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588286:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162790 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa55 expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588308:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572542.588310:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572542.588314:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162790 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa55 expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588518:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:0.0:1788572542.588587:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35910:x1875453360364800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 735us (1730us total) trans 4294970396 rc 301/301 00000100:00100000:0.0:1788572542.588595:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4112 00000100:00100000:0.0:1788572542.588619:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4113 00000100:00100000:0.0:1788572542.588626:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35911:x1875453360365056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.588635:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.588644:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1627a5 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa5c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588694:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 3/1,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588703:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588711:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.588719:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x83:0x0]: returning size 3 00010000:00010000:0.0:1788572542.588761:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588768:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588773:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588782:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.588785:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588791:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588799:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572542.588802:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572542.588805:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 2/0,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588810:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1627ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588833:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1627a5 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa5c expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588841:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572542.588843:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572542.588846:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1627a5 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa5c expref: 83 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.588856:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572542.588878:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35911:x1875453360365056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 252us (1829us total) trans 4294970397 rc 301/301 00000100:00100000:0.0:1788572542.588885:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4113 00000100:00100000:3.0:1788572542.591622:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.591699:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360365312 00000100:00100000:1.0:1788572542.591713:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360365312 00000100:00100000:1.0:1788572542.591779:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4114 00000100:00100000:1.0:1788572542.591791:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35912:x1875453360365312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.591813:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.591834:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1627ba lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa63 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.591955:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.591974:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.591986:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.591996:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x88:0x0]: returning size 3 00010000:00010000:1.0:1788572542.592135:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592143:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592148:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592154:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.592157:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592165:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592177:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:1.0:1788572542.592180:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572542.592185:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592191:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592237:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1627ba lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa63 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592247:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572542.592249:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572542.592253:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1627ba lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa63 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.592261:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:1.0:1788572542.592326:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35912:x1875453360365312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 537us (712us total) trans 4294970398 rc 301/301 00000100:00100000:1.0:1788572542.592468:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4114 00000100:00100000:3.0:1788572542.598793:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.598830:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.598867:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.598986:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360365440 00000100:00100000:0.0:1788572542.598996:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360365440 00000100:00100000:0.0:1788572542.599079:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360365568 00000100:00100000:0.0:1788572542.599083:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360365568 00000100:00100000:0.0:1788572542.599098:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360365696 00000100:00100000:0.0:1788572542.599100:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360365696 00000100:00100000:0.0:1788572542.599117:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4115 00000100:00100000:0.0:1788572542.599125:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35911:x1875453360365440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572542.599158:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x83:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572542.599283:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35911:x1875453360365440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 157us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.599291:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4115 00000100:00100000:0.0:1788572542.599312:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4116 00000100:00100000:0.0:1788572542.599319:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35910:x1875453360365568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572542.599334:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x89:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572542.599372:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35910:x1875453360365568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 54us (541us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.599378:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4116 00000100:00100000:0.0:1788572542.599390:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4117 00000100:00100000:0.0:1788572542.599396:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35912:x1875453360365696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572542.599408:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x88:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572542.599443:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35912:x1875453360365696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 48us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.599449:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4117 00000100:00100000:3.0:1788572542.603517:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.603619:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360365824 00000100:00100000:0.0:1788572542.603630:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360365824 00000100:00100000:0.0:1788572542.603702:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4118 00000100:00100000:0.0:1788572542.603714:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35910:x1875453360365824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.603734:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.603758:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1627cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa6a expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.603835:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.603856:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.603868:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.603883:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.603891:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.603898:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604352:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1627e4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604362:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1627e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604369:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1627e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.604401:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572542.604405:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572542.604427:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1627e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604439:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604445:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604462:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604469:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.604474:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604482:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604495:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572542.604498:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.604504:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604510:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1627d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604525:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604535:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604541:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604547:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.604550:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604557:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604564:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627fb1e7f00) 00010000:00010000:0.0:1788572542.604566:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.604569:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604575:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1627dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604607:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1627cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa6a expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604618:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1627e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa6a expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.604624:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572542.604626:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572542.604630:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:0.0:1788572542.604698:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35910:x1875453360365824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 986us (1195us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.604707:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4118 00000100:00100000:0.0:1788572542.612266:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.612300:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.613978:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360366080 00000100:00100000:1.0:1788572542.613996:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360366080 00000100:00100000:1.0:1788572542.614064:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360366208 00000100:00100000:1.0:1788572542.614068:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360366208 00000100:00100000:1.0:1788572542.614155:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4119 00000100:00100000:1.0:1788572542.614167:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35911:x1875453360366080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.614191:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.614194:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572542.614233:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1627e4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa6a expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.614262:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.614268:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572542.614284:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627eb lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa71 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572542.614879:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x49:0x0], index 1: 0 00010000:00010000:1.0:1788572542.615123:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1627f2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615138:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1627f2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615148:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1627f2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615211:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1627eb lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa71 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615226:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1627f2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa71 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615234:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572542.615236:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572542.615239:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:1.0:1788572542.615310:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35911:x1875453360366080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1145us (3055us total) trans 4294970399 rc 0/0 00000100:00100000:1.0:1788572542.615317:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4119 00000100:00100000:1.0:1788572542.615340:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4120 00000100:00100000:1.0:1788572542.615348:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35912:x1875453360366208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.615359:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.615371:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162800 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa78 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615414:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615424:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615433:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615445:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615449:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615467:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615704:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162815 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615714:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162815 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615721:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162815 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572542.615741:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:1.0:1788572542.615745:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572542.615755:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162815 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615770:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615777:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615783:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615789:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.615793:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615800:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615809:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572542.615812:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.615816:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615821:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162807 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615830:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615837:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615843:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615848:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.615851:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615857:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615865:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627d0ee2000) 00010000:00010000:1.0:1788572542.615868:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.615870:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615876:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb16280e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615904:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162800 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa78 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615916:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162815 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa78 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.615923:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.615925:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572542.615928:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:1.0:1788572542.615956:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35912:x1875453360366208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 609us (3653us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.615963:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4120 00000100:00100000:3.0:1788572542.624121:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.624661:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360366464 00000100:00100000:2.0:1788572542.624673:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360366464 00000100:00100000:2.0:1788572542.624743:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4121 00000100:00100000:2.0:1788572542.624753:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35910:x1875453360366464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.624779:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.624783:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572542.624809:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162815 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa78 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.624827:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572542.624832:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572542.624895:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb16281c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa7f expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572542.625446:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4a:0x0], index 0: 0 00010000:00010000:2.0:1788572542.625667:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162823 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.625681:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162823 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.625692:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162823 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.625756:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb16281c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa7f expref: 88 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.625774:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162823 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa7f expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.625781:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572542.625785:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572542.625787:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678200, rc 0) 00000100:00100000:2.0:1788572542.626043:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35910:x1875453360366464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1291us (1931us total) trans 4294970400 rc 0/0 00000100:00100000:2.0:1788572542.626053:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4121 00000100:00100000:2.0:1788572542.630924:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.630988:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.631141:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360366592 00000100:00100000:0.0:1788572542.631153:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360366592 00000100:00100000:0.0:1788572542.631227:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360366848 00000100:00100000:0.0:1788572542.631230:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360366848 00000100:00100000:0.0:1788572542.631256:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4122 00000100:00100000:0.0:1788572542.631266:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35910:x1875453360366592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.631290:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.631311:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162831 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa86 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631380:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631394:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631406:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631419:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631426:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631436:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631743:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162846 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631751:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162846 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631759:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162846 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.631785:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572542.631789:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572542.631808:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162846 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631819:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631825:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631832:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.631838:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.631842:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.632038:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4123 00000100:00100000:1.0:1788572542.632049:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35912:x1875453360366848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.632059:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.632068:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16284d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa8d expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572542.632414:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4a:0x0], index 1: 0 00010000:00010000:1.0:1788572542.632605:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162854 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.632616:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162854 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.632624:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162854 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.632680:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb16284d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa8d expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.632697:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162854 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa8d expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.632705:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572542.632709:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572542.632712:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:1.0:1788572542.632774:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35912:x1875453360366848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 727us (1786us total) trans 4294970401 rc 0/0 00000100:00100000:1.0:1788572542.632782:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4123 00010000:00010000:0.0:1788572542.633147:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633158:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572542.633160:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.633165:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633171:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162838 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633185:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633191:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633197:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633404:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.633410:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633417:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633426:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627fb1e7f00) 00010000:00010000:0.0:1788572542.633429:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.633432:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633438:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16283f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633486:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162831 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa86 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633504:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162846 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa86 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.633511:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572542.633513:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572542.633516:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572542.633546:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35910:x1875453360366592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2280us (2632us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.633552:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4122 00000100:00100000:3.0:1788572542.638138:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.638424:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360367104 00000100:00100000:0.0:1788572542.638434:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360367104 00000100:00100000:0.0:1788572542.638504:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4124 00000100:00100000:0.0:1788572542.638514:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35912:x1875453360367104:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.638575:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788572542.638599:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162854 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa8d expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638615:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:0.0:1788572542.638631:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162846 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa86 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638643:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572542.638647:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572542.638669:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638683:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638692:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638708:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638715:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.638720:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639091:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639099:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639105:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639110:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.639114:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639121:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639130:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572542.639133:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.639136:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639142:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162862 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639150:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639156:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639161:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639165:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.639167:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639172:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639178:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xdbd61c03 (ffff9627fb1e7300) 00010000:00010000:0.0:1788572542.639180:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.639182:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.639187:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162869 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0xdbd61c03 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.639419:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35912:x1875453360367104:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 904us (1292us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572542.639428:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4124 00000100:00100000:0.0:1788572542.642743:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.643020:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360367232 00000100:00100000:1.0:1788572542.643031:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360367232 00000100:00100000:1.0:1788572542.643087:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 288 00000100:00100000:1.0:1788572542.643097:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:2349:x1875453360367232:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572542.643149:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.643167:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162870 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efa94 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643238:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643253:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643264:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643282:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643289:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643296:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643490:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643498:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643504:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643511:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.643515:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643522:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643535:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.643538:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.643543:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643549:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162877 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643561:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643566:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643571:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643576:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.643578:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643584:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643594:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xf72cb2f5 (ffff9627d0ee2200) 00010000:00010000:1.0:1788572542.643597:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.643599:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643605:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb16287e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xf72cb2f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643635:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162870 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa94 expref: 89 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643644:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.643646:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572542.643650:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162870 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa94 expref: 89 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.643659:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc -2) 00000100:00100000:1.0:1788572542.643723:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:2349:x1875453360367232:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 627us (990us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572542.643732:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 288 00000100:00100000:0.0:1788572542.648632:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.648733:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360367360 00000100:00100000:1.0:1788572542.648745:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360367360 00000100:00100000:1.0:1788572542.648818:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4125 00000100:00100000:1.0:1788572542.648829:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35912:x1875453360367360:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.648851:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.648878:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162885 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efacc expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.648939:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.648954:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.648966:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.649112:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162885 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.649132:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.649141:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572542.649145:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572542.649148:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:1.0:1788572542.649201:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35912:x1875453360367360:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 375us (582us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.649811:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4125 00000100:00100000:3.0:1788572542.652572:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.652624:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.652663:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360367744 00000100:00100000:3.0:1788572542.652673:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360367744 00000100:00100000:3.0:1788572542.652728:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4126 00000100:00100000:3.0:1788572542.652737:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35911:x1875453360367744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.652798:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572542.652828:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1627f2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa71 expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.652853:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627d0ee2300) 00010000:00010000:3.0:1788572542.652865:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572542.652891:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.652904:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.652913:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.652933:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.652939:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.652946:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653190:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653199:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653277:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653284:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.653288:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653295:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653304:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572542.653307:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.653310:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653316:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162893 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653326:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653332:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653338:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653344:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572542.653347:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653353:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653360:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x45e17595 (ffff9627ffc51000) 00010000:00010000:3.0:1788572542.653363:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572542.653365:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.653371:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16289a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x45e17595 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.653443:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35911:x1875453360367744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 707us (818us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572542.653451:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4126 00000100:00100000:1.0:1788572542.654571:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360367488 00000100:00100000:1.0:1788572542.654583:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360367488 00000100:00100000:1.0:1788572542.654644:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 289 00000100:00100000:1.0:1788572542.654654:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c97cc850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:2350:x1875453360367488:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572542.654699:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572542.654715:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1628a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efad3 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654765:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654776:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654786:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654802:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654808:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654814:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654985:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654993:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.654999:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655028:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.655032:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655044:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655054:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.655057:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.655061:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655067:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1628a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655080:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655085:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655091:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655096:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.655098:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655104:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655111:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xa7c11174 (ffff9627ffc16900) 00010000:00010000:1.0:1788572542.655113:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.655116:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655125:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628af lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xa7c11174 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655150:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1628a1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efad3 expref: 89 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655159:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572542.655161:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572542.655165:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1628a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efad3 expref: 89 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.655173:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad600, rc -2) 00000100:00100000:1.0:1788572542.655237:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c97cc850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:2350:x1875453360367488:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 585us (2677us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572542.655244:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 289 00000100:00100000:0.0:1788572542.660040:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.660084:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.660111:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.660136:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360368000 00000100:00100000:0.0:1788572542.660146:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360368000 00000100:00100000:0.0:1788572542.660201:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360368256 00000100:00100000:0.0:1788572542.660219:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360368256 00000100:00100000:0.0:1788572542.660240:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360368384 00000100:00100000:0.0:1788572542.660243:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360368384 00000100:00100000:0.0:1788572542.660262:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4127 00000100:00100000:0.0:1788572542.660273:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35912:x1875453360368000:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.660371:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.660388:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.660400:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572542.660813:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].59, 88 off12976 00000004:00080000:0.0:1788572542.660823:0:5839:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 890729818721ns, next at 891729815274ns, rc = 0 00000004:00010000:0.0:1788572542.660912:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000069f7cb4d reply state 00000000f655ed17 transno 4294970402 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.660922:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572542.661041:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35912:x1875453360368000:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 768us (1010us total) trans 4294970402 rc 0/0 00000100:00100000:0.0:1788572542.661048:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4127 00000100:00100000:0.0:1788572542.661072:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4128 00000100:00100000:0.0:1788572542.661079:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35910:x1875453360368256:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.661104:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.661122:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162823 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efa7f expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661137:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:0.0:1788572542.661146:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572542.661159:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661167:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661174:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661187:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661194:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661199:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661307:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661315:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661322:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661327:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.661331:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661339:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661348:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572542.661351:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.661355:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661360:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1628bd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661372:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661378:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661383:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661388:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.661391:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661396:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661403:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x6b840ae3 (ffff9627fb1e7f00) 00010000:00010000:0.0:1788572542.661405:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.661407:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661413:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1628c4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x6b840ae3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.661465:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35910:x1875453360368256:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 373us (1367us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572542.661472:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4128 00000100:00100000:0.0:1788572542.661486:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4129 00000100:00100000:0.0:1788572542.661493:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35911:x1875453360368384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.661502:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.661512:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1628cb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb0b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661536:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661545:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661553:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661619:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1628cb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661634:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.661641:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572542.661644:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572542.661647:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572542.661679:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35911:x1875453360368384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 187us (1567us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.661686:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4129 00010000:00010000:2.0:1788572542.671917:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.671949:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.671957:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.671963:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.671970:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.671979:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.671999:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:2.0:1788572542.672035:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572542.672041:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.672047:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1628b6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.672753:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.673095:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360368512 00000100:00100000:2.0:1788572542.673106:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360368512 00000100:00100000:2.0:1788572542.673157:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4130 00000100:00100000:2.0:1788572542.673166:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35912:x1875453360368512:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.673191:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.673218:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb12 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.673278:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.673292:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.673310:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:2.0:1788572542.673313:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572542.673316:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678400, rc 0) 00000100:00100000:2.0:1788572542.673372:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35912:x1875453360368512:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 206us (625us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.673379:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4130 00000100:00100000:0.0:1788572542.676210:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.676273:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360368768 00000100:00100000:2.0:1788572542.676283:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360368768 00000100:00100000:2.0:1788572542.676338:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4131 00000100:00100000:2.0:1788572542.676347:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35910:x1875453360368768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.676368:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.676393:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628e0 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb19 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.676428:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.676441:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.676451:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.676555:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628e0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.676572:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.676581:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.676584:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572542.676588:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679400, rc 0) 00000100:00100000:2.0:1788572542.676646:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35910:x1875453360368768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 301us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.676653:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4131 00000100:00100000:1.0:1788572542.679098:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.679217:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360368640 00000100:00100000:0.0:1788572542.679230:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360368640 00000100:00100000:0.0:1788572542.679282:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1089 00000100:00100000:0.0:1788572542.679291:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360368640:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.679619:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360368640:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 328us (529us total) trans 4294967581 rc 0/0 00000100:00100000:0.0:1788572542.679633:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1089 00000100:00100000:1.0:1788572542.685648:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.685741:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.686508:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360369024 00000100:00100000:2.0:1788572542.686521:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360369024 00000100:00100000:2.0:1788572542.686631:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360369280 00000100:00100000:2.0:1788572542.686636:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360369280 00000100:00100000:2.0:1788572542.686668:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4132 00000100:00100000:2.0:1788572542.686678:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35910:x1875453360369024:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.686789:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,1 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.686810:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.686823:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.687072:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4133 00000100:00100000:1.0:1788572542.687081:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35911:x1875453360369280:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.687116:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.687125:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.687132:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572542.687546:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].60, 88 off13064 00000004:00010000:1.0:1788572542.687653:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002c965793 reply state 000000006b13bed1 transno 4294970403 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.687664:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572542.687784:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35911:x1875453360369280:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 703us (2041us total) trans 4294970403 rc 0/0 00000100:00100000:1.0:1788572542.687793:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4133 00000040:00080000:2.0:1788572542.689216:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].62, 88 off13168 00000004:00080000:2.0:1788572542.689233:0:5837:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 890758227660ns, next at 891758224557ns, rc = 0 00000004:00010000:2.0:1788572542.689330:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dc6a75ee reply state 00000000802098b8 transno 4294970404 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.689342:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572542.689445:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35910:x1875453360369024:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2765us (3804us total) trans 4294970404 rc 0/0 00000100:00100000:2.0:1788572542.689467:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4132 00000100:00100000:2.0:1788572542.690751:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.690892:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.690924:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360369664 00000100:00100000:2.0:1788572542.690941:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360369664 00000100:00100000:2.0:1788572542.691024:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4134 00000100:00100000:2.0:1788572542.691034:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35911:x1875453360369664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.691056:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.691078:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb20 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691141:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 96 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691157:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 96 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691175:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572542.691178:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572542.691182:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3714000, rc 0) 00000100:00100000:2.0:1788572542.691225:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35911:x1875453360369664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 193us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.691232:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4134 00010000:00010000:2.0:1788572542.691292:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691301:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691307:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691318:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.691323:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691331:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691344:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572542.691346:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572542.691350:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.691356:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb1628f5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.694121:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360369536 00000100:00100000:1.0:1788572542.694141:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360369536 00000100:00100000:1.0:1788572542.694308:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 896 00000100:00100000:1.0:1788572542.694320:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35912:x1875453360369536:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.694800:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35912:x1875453360369536:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 481us (4054us total) trans 4294970405 rc 0/0 00000100:00100000:1.0:1788572542.694809:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 896 00010000:00010000:2.0:1788572542.695667:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.695682:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.695690:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.695697:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.695702:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.695712:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.695734:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.695737:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572542.695743:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 2/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.695749:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1628ee lrc: 0/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.695801:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.697359:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360369792 00000100:00100000:0.0:1788572542.697374:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360369792 00000100:00100000:0.0:1788572542.697438:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4135 00000100:00100000:0.0:1788572542.697449:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35910:x1875453360369792:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.697482:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.697504:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb27 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.697566:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.697580:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.697599:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572542.697601:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572542.697604:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:0.0:1788572542.697672:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35910:x1875453360369792:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 225us (1872us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.697680:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4135 00000100:00100000:2.0:1788572542.699869:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.699921:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.700038:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360370048 00000100:00100000:0.0:1788572542.700048:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360370048 00000100:00100000:0.0:1788572542.700109:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4136 00000100:00100000:0.0:1788572542.700119:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35912:x1875453360370048:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.700243:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35912:x1875453360370048:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 124us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.700251:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4136 00000100:00100000:3.0:1788572542.700727:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360370176 00000100:00100000:3.0:1788572542.700733:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360370176 00000100:00100000:3.0:1788572542.700782:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1090 00000100:00100000:3.0:1788572542.700791:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360370176:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.701185:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360370176:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 394us (1262us total) trans 4294967582 rc 0/0 00000100:00100000:3.0:1788572542.701193:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1090 00000100:00100000:2.0:1788572542.702921:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.703041:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360370304 00000100:00100000:1.0:1788572542.703052:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360370304 00000100:00100000:1.0:1788572542.703106:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1091 00000100:00100000:1.0:1788572542.703115:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360370304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.703420:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360370304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 304us (507us total) trans 4294967584 rc 0/0 00000100:00100000:1.0:1788572542.703428:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1091 00000100:00100000:0.0:1788572542.705922:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.706056:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360370688 00000100:00100000:3.0:1788572542.706066:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360370688 00000100:00100000:3.0:1788572542.706125:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 897 00000100:00100000:3.0:1788572542.706135:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35911:x1875453360370688:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.706724:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35911:x1875453360370688:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 589us (808us total) trans 4294970406 rc 0/0 00000100:00100000:3.0:1788572542.706734:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 897 00000100:00100000:2.0:1788572542.708262:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.708394:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.708530:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360370944 00000100:00100000:3.0:1788572542.708539:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360370944 00000100:00100000:3.0:1788572542.708597:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 898 00000100:00100000:3.0:1788572542.708606:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35910:x1875453360370944:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.709022:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35910:x1875453360370944:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 395us (605us total) trans 4294970407 rc 0/0 00000100:00100000:3.0:1788572542.709031:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 898 00000100:00100000:0.0:1788572542.710496:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360370432 00000100:00100000:0.0:1788572542.710505:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360370432 00000100:00100000:0.0:1788572542.710600:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4137 00000100:00100000:0.0:1788572542.710609:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35912:x1875453360370432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.710626:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.710646:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16290a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb2e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.710716:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162911 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.710734:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.710746:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572542.710783:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572542.710799:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.710825:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb16290a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb2e expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.710844:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162911 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb2e expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.710863:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572542.710866:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572542.710870:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:0.0:1788572542.710921:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35912:x1875453360370432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 313us (2666us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.710941:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4137 00000100:00100000:2.0:1788572542.713301:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.714434:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360371200 00000100:00100000:0.0:1788572542.714463:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360371200 00000100:00100000:0.0:1788572542.714538:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4138 00000100:00100000:0.0:1788572542.714548:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35911:x1875453360371200:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.714674:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35911:x1875453360371200:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 126us (1383us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.714682:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4138 00000100:00100000:0.0:1788572542.715619:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.715684:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360371456 00000100:00100000:2.0:1788572542.715691:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360371456 00000100:00100000:2.0:1788572542.715727:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4139 00000100:00100000:2.0:1788572542.715735:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35910:x1875453360371456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.715803:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35910:x1875453360371456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 69us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.715810:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4139 00000100:00100000:1.0:1788572542.717050:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.717173:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360371584 00000100:00100000:0.0:1788572542.717180:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360371584 00000100:00100000:0.0:1788572542.717230:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1092 00000100:00100000:0.0:1788572542.717239:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360371584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.717252:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.717272:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb35 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572542.717302:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737e, blocks: 0x1 00010000:00010000:0.0:1788572542.717319:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=890s v=(0 0) 00010000:00010000:0.0:1788572542.717324:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.717345:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.717364:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.717379:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572542.717432:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360371584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 193us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.717438:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1092 00000100:00100000:0.0:1788572542.723651:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.723708:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.723807:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360371712 00000100:00100000:3.0:1788572542.723819:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360371712 00000100:00100000:3.0:1788572542.723882:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360371840 00000100:00100000:3.0:1788572542.723885:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360371840 00000100:00100000:3.0:1788572542.723956:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4140 00000100:00100000:3.0:1788572542.723966:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35911:x1875453360371712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.723995:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.724047:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16291f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb3c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724117:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162926 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724134:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162926 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724150:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162926 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572542.724186:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572542.724205:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162926 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724230:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16291f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb3c expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724247:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162926 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb3c expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724256:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:3.0:1788572542.724258:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572542.724262:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572542.724314:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35911:x1875453360371712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 351us (680us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.724320:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4140 00000100:00100000:3.0:1788572542.724618:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4141 00000100:00100000:3.0:1788572542.724627:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35910:x1875453360371840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.724637:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.724646:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16292d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb43 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724681:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162934 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724695:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162934 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724703:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162934 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572542.724726:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572542.724735:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162934 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724752:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16292d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb43 expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724765:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162934 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb43 expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.724772:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572542.724774:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572542.724777:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc400, rc 0) 00000100:00100000:3.0:1788572542.724807:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35910:x1875453360371840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 182us (1098us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.724815:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4141 00000100:00100000:0.0:1788572542.726981:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.727709:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.729306:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360372096 00000100:00100000:2.0:1788572542.729322:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360372096 00000100:00100000:0.0:1788572542.730133:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.730239:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4142 00000100:00100000:2.0:1788572542.730250:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35912:x1875453360372096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.730382:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35912:x1875453360372096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 132us (3464us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.730390:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4142 00000100:00100000:3.0:1788572542.731268:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360372224 00000100:00100000:3.0:1788572542.731278:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360372224 00000100:00100000:2.0:1788572542.731752:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360372352 00000100:00100000:2.0:1788572542.731758:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360372352 00000100:00100000:2.0:1788572542.731789:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1093 00000100:00100000:2.0:1788572542.731798:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360372224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.731810:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.731829:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb4a expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572542.731856:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x49:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737e, blocks: 0x1 00010000:00010000:2.0:1788572542.731873:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=890s v=(0 0) 00010000:00010000:2.0:1788572542.731877:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.731887:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.731905:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.731918:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0030e00, rc 0) 00000100:00100000:2.0:1788572542.731974:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360372224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 177us (4267us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.731980:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1093 00000100:00100000:2.0:1788572542.731998:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1094 00000100:00100000:2.0:1788572542.732019:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360372352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.732030:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.732043:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb51 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572542.732059:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737e, blocks: 0x1 00010000:00010000:2.0:1788572542.732069:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=890s v=(0 0) 00010000:00010000:2.0:1788572542.732072:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.732077:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.732088:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.732096:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031c00, rc 0) 00000100:00100000:2.0:1788572542.732115:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360372352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 111us (1987us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.732120:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1094 00000100:00100000:0.0:1788572542.735932:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.736047:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360372480 00000100:00100000:3.0:1788572542.736057:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360372480 00000100:00100000:3.0:1788572542.736123:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1095 00000100:00100000:3.0:1788572542.736133:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35912:x1875453360372480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.736154:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572542.736173:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb58 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.736204:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.736213:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572542.736218:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572542.736224:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.736264:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.736280:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.736297:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382354304:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.736386:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5e00f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.738444:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.738543:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360372736 00000100:00100000:2.0:1788572542.738552:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360372736 00000100:00100000:2.0:1788572542.738601:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4143 00000100:00100000:2.0:1788572542.738610:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35910:x1875453360372736:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.738724:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35910:x1875453360372736:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 114us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.738733:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4143 00000100:00100000:0.0:1788572542.740727:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.740840:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360372992 00000100:00100000:3.0:1788572542.740849:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360372992 00000100:00100000:3.0:1788572542.740911:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4144 00000100:00100000:3.0:1788572542.740921:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35911:x1875453360372992:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.741068:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35911:x1875453360372992:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 149us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.741076:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4144 00000100:00100000:1.0:1788572542.741169:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382354304:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.741201:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.741224:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:1.0:1788572542.741252:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35912:x1875453360372480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5121us (5331us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.741259:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1095 00000100:00100000:1.0:1788572542.747200:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.747449:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373120 00000100:00100000:2.0:1788572542.747471:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373120 00010000:00010000:2.0:1788572542.747514:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.747559:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 979 00000100:00100000:2.0:1788572542.747569:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360373120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572542.747585:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572542.747609:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.747623:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.747645:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8077b05/0x85ace67feb162918 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb35 expref: 15 pid: 20911 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.747664:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.747668:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.747675:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.747685:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.747724:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.747744:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36ddf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382354432:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572542.747829:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.751767:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.754791:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36ddf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382354432:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572542.754819:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.754879:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360373120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7310us (7688us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.754891:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 979 00000100:00100000:1.0:1788572542.756182:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.757497:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.760734:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373504 00000100:00100000:1.0:1788572542.760753:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373504 00002000:00100000:1.0:1788572542.760820:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4a:0x0] (0->2) req@ffff9627fb470700 x1875453360373504/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.760833:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb470700 with x1875453360373504 ext(0->2) 00002000:00010000:1.0:1788572542.760838:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb470700 00000100:00100000:1.0:1788572542.760886:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 251 00000100:00100000:1.0:1788572542.760896:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb470700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360373504:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.761405:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb470700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360373504:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 509us (3913us total) trans 4294967583 rc 0/0 00000100:00100000:1.0:1788572542.761414:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 251 00002000:00100000:1.0:1788572542.761421:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4a:0x0] (0->2) req@ffff9627fb470700 x1875453360373504/t4294967583(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/448 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572542.761430:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb470700 with x1875453360373504 ext(0->2) 00002000:00010000:1.0:1788572542.761433:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb470700 00000100:00100000:1.0:1788572542.764862:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.764978:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373632 00000100:00100000:0.0:1788572542.764988:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373632 00000100:00100000:0.0:1788572542.765067:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4145 00000100:00100000:0.0:1788572542.765077:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35912:x1875453360373632:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.768422:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373248 00000100:00100000:3.0:1788572542.768438:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373248 00000100:00100000:2.0:1788572542.770128:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373376 00000100:00100000:2.0:1788572542.770144:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373376 00000100:00100000:2.0:1788572542.770261:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1096 00000100:00100000:2.0:1788572542.770273:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35911:x1875453360373248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.770303:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572542.770323:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb5f expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.770369:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.770379:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.770384:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572542.770390:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.770432:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.770448:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.770475:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382354560:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.770572:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.776654:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1097 00000100:00100000:0.0:1788572542.777201:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35910:x1875453360373376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.777223:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572542.777243:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb66 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.777277:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.777291:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.777296:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.777302:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.777345:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.777358:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.777374:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb473b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382354688:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572542.777472:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb180 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.778414:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3305:1875453382355072:0@lo:400:kworker.0 00000100:00100000:2.0:1788572542.778483:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.778515:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572542.778521:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3305:1875453382354944:0@lo:400:kworker.0 00000100:00100000:2.0:1788572542.778542:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.778570:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382355072 00000100:00100000:2.0:1788572542.778576:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382355072 00000100:00100000:2.0:1788572542.778620:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382354944 00000100:00100000:2.0:1788572542.778623:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382354944 00000100:00100000:3.0:1788572542.778762:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382355200:0@lo:400:kworker.0 00000100:00100000:3.0:1788572542.778795:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572542.778813:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788572542.778819:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3302:1875453382354816:0@lo:400:kworker.0 00000100:00100000:3.0:1788572542.778845:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572542.778879:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382354816 00000100:00100000:3.0:1788572542.778883:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382354816 00000100:00100000:3.0:1788572542.778914:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:3.0:1788572542.778921:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efe450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3302:x1875453382354816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572542.778966:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efe450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3302:x1875453382354816:12345-0@lo:400:kworker.0 Request processed in 46us (120us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.778973:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00100000:3.0:1788572542.779038:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3302:1875453382354816:0@lo:400:kworker.0 00000100:00100000:3.0:1788572542.779060:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382355200 00000100:00100000:3.0:1788572542.779063:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382355200 00000100:00100000:3.0:1788572542.779083:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:3.0:1788572542.779089:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1875453382355072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572542.779104:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:1.0:1788572542.779113:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1875453382354944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572542.779125:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1875453382355072:12345-0@lo:400:kworker.0 Request processed in 36us (644us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.779131:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1788572542.779146:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1875453382354944:12345-0@lo:400:kworker.0 Request processed in 35us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.779146:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1788572542.779152:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1788572542.779152:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382355200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572542.779180:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382355200:12345-0@lo:400:kworker.0 Request processed in 28us (384us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.779187:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1788572542.779201:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382355200:0@lo:400:kworker.0 00000100:00080000:2.0:1788572542.779249:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572542.779274:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3305:1875453382354944:0@lo:400:kworker.0 00000100:00100000:2.0:1788572542.779284:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3305:1875453382355072:0@lo:400:kworker.0 00000100:00100000:1.0:1788572542.781700:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.781773:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382354560:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572542.781811:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.781834:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031600, rc 0) 00000100:00100000:3.0:1788572542.781894:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35911:x1875453360373248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 11622us (30137us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.781903:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1096 00000100:00100000:0.0:1788572542.782451:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373760 00000100:00100000:0.0:1788572542.782470:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373760 00010000:00010000:0.0:1788572542.782507:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.782551:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 980 00000100:00100000:0.0:1788572542.782559:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e51050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2349:x1875453360373760:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.782572:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.782595:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.782613:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.782636:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d2456d5a/0x85ace67feb16293b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb4a expref: 17 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.782663:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.782667:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.782676:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.782685:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.782721:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.782735:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382355328:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572542.782794:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.786067:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb473b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382354688:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572542.786107:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.786128:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:3.0:1788572542.786188:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35910:x1875453360373376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9521us (30011us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.786197:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1097 00000100:00100000:0.0:1788572542.788556:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.788907:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360373888 00000100:00100000:1.0:1788572542.788917:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360373888 00010000:00010000:1.0:1788572542.788954:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.788994:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 981 00000100:00100000:1.0:1788572542.789024:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360373888:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.789040:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.789065:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.789078:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.789101:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319fba8f/0x85ace67feb162942 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb51 expref: 14 pid: 6681 timeout: 990 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.789118:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.789123:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572542.789132:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.789142:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.789181:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.789195:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcfb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382355456:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572542.789283:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.790361:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382355328:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572542.790383:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.790421:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e51050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360373760:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7862us (8733us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.790433:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 980 00000100:00100000:1.0:1788572542.797641:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.797775:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374016 00000100:00100000:0.0:1788572542.797787:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374016 00002000:00100000:0.0:1788572542.797843:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x49:0x0] (0->2) req@ffff9626c1bcc000 x1875453360374016/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.797855:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bcc000 with x1875453360374016 ext(0->2) 00002000:00010000:0.0:1788572542.797862:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1bcc000 00000100:00100000:0.0:1788572542.797900:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 252 00000100:00100000:0.0:1788572542.797910:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360374016:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.798608:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360374016:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 695us (975us total) trans 4294967584 rc 0/0 00000100:00100000:0.0:1788572542.798617:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 252 00002000:00100000:0.0:1788572542.798625:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x49:0x0] (0->2) req@ffff9626c1bcc000 x1875453360374016/t4294967584(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/448 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.798632:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bcc000 with x1875453360374016 ext(0->2) 00002000:00010000:0.0:1788572542.798636:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1bcc000 00000100:00100000:0.0:1788572542.802038:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcfb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382355456:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572542.802069:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.802121:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360373888:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 13118us (13573us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.802131:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 981 00000100:00100000:3.0:1788572542.807054:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.807142:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374144 00000100:00100000:0.0:1788572542.807153:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374144 00002000:00100000:0.0:1788572542.807303:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4a:0x0] (0->2) req@ffff9626c10a1180 x1875453360374144/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/0 e 0 to 0 dl 1788572553 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.807325:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a1180 with x1875453360374144 ext(0->2) 00002000:00010000:0.0:1788572542.807445:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a1180 00000100:00100000:0.0:1788572542.807689:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 253 00000100:00100000:0.0:1788572542.807704:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360374144:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.808760:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360374144:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 1055us (1713us total) trans 4294967585 rc 0/0 00000100:00100000:0.0:1788572542.808770:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 253 00002000:00100000:0.0:1788572542.808778:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4a:0x0] (0->2) req@ffff9626c10a1180 x1875453360374144/t4294967585(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:203/0 lens 496/448 e 0 to 0 dl 1788572553 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572542.808788:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a1180 with x1875453360374144 ext(0->2) 00002000:00010000:0.0:1788572542.808791:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a1180 00000100:00100000:1.0:1788572542.811031:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.811388:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374272 00000100:00100000:2.0:1788572542.811398:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374272 00000100:00100000:2.0:1788572542.811465:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4146 00000100:00100000:2.0:1788572542.811476:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35911:x1875453360374272:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572542.811623:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35911:x1875453360374272:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 149us (631us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572542.811631:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4146 00000100:00100000:0.0:1788572542.820736:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.820805:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374400 00000100:00100000:2.0:1788572542.820816:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374400 00000100:00100000:2.0:1788572542.820879:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1098 00000100:00100000:2.0:1788572542.820891:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2347:x1875453360374400:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.823545:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.823697:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374528 00000100:00100000:1.0:1788572542.823707:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374528 00000100:00100000:1.0:1788572542.823766:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4147 00000100:00100000:1.0:1788572542.823776:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35910:x1875453360374528:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.823908:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35910:x1875453360374528:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 133us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.823915:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4147 00000100:00100000:3.0:1788572542.835525:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35912:x1875453360373632:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 70437us (70663us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.835543:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4145 00000001:00080000:2.0:1788572542.835697:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970402 is committed 00000004:00080000:2.0:1788572542.835704:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 36 committed 00000020:00080000:2.0:1788572542.835740:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c7f0fa80 refcount 2 committed 0 result 0 batchid 109 00000020:00080000:2.0:1788572542.835744:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cea600 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c7f0f180 00000020:00080000:2.0:1788572542.835747:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceab00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff962800914000 00000001:00080000:2.0:1788572542.835771:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970405 is committed 00000020:00080000:2.0:1788572542.835797:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 108 batchid updated 00000020:00080000:0.0:1788572542.835800:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 109 committed batchid 105 00000004:00080000:2.0:1788572542.835804:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 41 committed 00000020:00080000:2.0:1788572542.835821:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627e7495000 refcount 2 committed 0 result 0 batchid 111 00000020:00080000:2.0:1788572542.835824:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc300 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9628021586c0 00000020:00080000:2.0:1788572542.835826:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffd0e400 00000001:00080000:2.0:1788572542.835840:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970407 is committed 00000020:00080000:2.0:1788572542.835848:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c4c47cc0 refcount 2 committed 0 result 0 batchid 110 00000020:00080000:2.0:1788572542.835851:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de400 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff5483c0 00000020:00080000:2.0:1788572542.835853:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627fb49d800 00000020:00080000:0.0:1788572542.835995:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 111 committed batchid 108 00000020:00080000:0.0:1788572542.836041:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 108 00000100:00100000:0.0:1788572542.836059:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382355584:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572542.836070:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382355840:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572542.836191:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.836225:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1788572542.836232:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382355712:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572542.836256:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.836273:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382356224:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572542.836292:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572542.836319:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382355840 00000100:00100000:2.0:1788572542.836327:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382355840 00000100:00100000:2.0:1788572542.836360:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382355712 00000100:00100000:2.0:1788572542.836363:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382355712 00000100:00100000:2.0:1788572542.836381:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382356224 00000100:00100000:2.0:1788572542.836384:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382356224 00000100:00100000:2.0:1788572542.836399:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1099 00000100:00100000:2.0:1788572542.836407:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382355840:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572542.836411:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572542.836430:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382356096:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1788572542.836450:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572542.836472:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788572542.836477:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382355968:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1788572542.836495:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572542.836515:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382355584 00000100:00100000:0.0:1788572542.836519:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382355584 00000100:00100000:0.0:1788572542.836535:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382356096 00000100:00100000:0.0:1788572542.836537:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382356096 00000100:00100000:0.0:1788572542.836544:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382355968 00000100:00100000:0.0:1788572542.836546:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382355968 00000100:00100000:0.0:1788572542.836555:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1100 00000100:00100000:0.0:1788572542.836562:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382355712:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1788572542.836593:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572542.836597:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb mtime from disk: 0 -> 1788572542 00002000:00010000:2.0:1788572542.836600:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1788572542.836602:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb ctime from disk: 1788572542 -> 1788572542 00002000:00010000:2.0:1788572542.836605:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572542.836650:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382355712:12345-0@lo:2:osp-syn-1-0.0 Request processed in 89us (393us total) trans 4294967586 rc 0/0 00000100:00100000:0.0:1788572542.836657:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1100 00000100:00100000:0.0:1788572542.836673:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1101 00000100:00100000:0.0:1788572542.836680:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382356224:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:0.0:1788572542.836774:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572542.836777:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb mtime from disk: 0 -> 1788572542 00002000:00010000:0.0:1788572542.836779:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1788572542.836780:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb ctime from disk: 1788572542 -> 1788572542 00002000:00010000:0.0:1788572542.836783:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1788572542.836807:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382356224:12345-0@lo:2:osp-syn-0-0.0 Request processed in 128us (513us total) trans 4294967586 rc 0/0 00000100:00100000:0.0:1788572542.836813:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1101 00000100:00100000:0.0:1788572542.836824:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1102 00000100:00100000:0.0:1788572542.836830:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382355584:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572542.836914:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1103 00000100:00100000:0.0:1788572542.836917:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382355584:12345-0@lo:2:osp-syn-1-0.0 Request processed in 88us (505us total) trans 4294967587 rc 0/0 00000100:00100000:3.0:1788572542.836922:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382356096:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1788572542.836923:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1102 00000100:00100000:0.0:1788572542.836934:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1104 00000100:00100000:0.0:1788572542.836939:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382355968:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572542.837086:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382355840:12345-0@lo:2:osp-syn-1-0.0 Request processed in 232us (447us total) trans 4294967585 rc 0/0 00000100:00100000:2.0:1788572542.837095:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1099 00000004:00080000:2.0:1788572542.837144:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b64a80/2, rc 0, transno 4294967586 00000100:00100000:2.0:1788572542.837159:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382355712:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1788572542.837171:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b64700/2, rc 0, transno 4294967586 00000100:00100000:2.0:1788572542.837175:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382356224:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572542.837182:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b67800/2, rc 0, transno 4294967585 00000100:00100000:2.0:1788572542.837186:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382355840:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572542.837311:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+11:3303:x1875453382355968:12345-0@lo:2:osp-syn-0-0.0 Request processed in 371us (813us total) trans 4294967588 rc 0/0 00000100:00100000:0.0:1788572542.837317:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1104 00000004:00080000:0.0:1788572542.837548:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b66680/2, rc 0, transno 4294967588 00000100:00100000:0.0:1788572542.837554:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382355968:0@lo:2:osp-syn-0-0.0 00000004:00080000:0.0:1788572542.837563:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b66300/2, rc 0, transno 4294967587 00000100:00100000:0.0:1788572542.837567:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382355584:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572542.838807:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.838902:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374656 00000100:00100000:0.0:1788572542.838910:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374656 00000100:00100000:0.0:1788572542.838955:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1105 00000100:00100000:0.0:1788572542.838964:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360374656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572542.840046:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.840356:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360374784 00000100:00100000:0.0:1788572542.840363:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360374784 00000100:00100000:0.0:1788572542.840408:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1106 00000100:00100000:0.0:1788572542.840416:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360374784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572542.840482:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382356096:12345-0@lo:2:osp-syn-0-0.0 Request processed in 3560us (4029us total) trans 4294967587 rc 0/0 00000100:00100000:3.0:1788572542.840490:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1103 00000004:00080000:0.0:1788572542.840502:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b67b80/2, rc 0, transno 4294967587 00000100:00100000:0.0:1788572542.840507:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382356096:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1788572542.874869:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2347:x1875453360374400:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 53968us (54135us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.874885:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1098 00000001:00080000:0.0:1788572542.874934:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967584 is committed 00000001:00080000:0.0:1788572542.874950:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967580 is committed 00000100:00100000:3.0:1788572542.881235:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.881792:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360375040 00000100:00100000:0.0:1788572542.881806:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360375040 00000100:00100000:0.0:1788572542.882757:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4148 00000100:00100000:0.0:1788572542.882773:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35911:x1875453360375040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.882829:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.882857:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162926 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb3c expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.882878:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572542.882883:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572542.882915:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.882929:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.882940:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572542.883154:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e930ce8c reply state 00000000440baa7f transno 4294970408 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.883164:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572542.883303:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35911:x1875453360375040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 522us (2068us total) trans 4294970408 rc 0/0 00000100:00100000:0.0:1788572542.883312:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4148 00010000:00010000:1.0:1788572542.889848:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.889864:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.889876:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.889882:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.889887:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.889897:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.889916:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572542.889920:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572542.889925:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.889932:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16295e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.898501:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.898598:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360375168 00000100:00100000:1.0:1788572542.898608:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360375168 00000100:00100000:1.0:1788572542.898698:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1107 00000100:00100000:1.0:1788572542.898708:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360375168:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572542.898980:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x49:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572542.898984:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x49:0x0] updating lvb mtime from disk: 0 -> 1788572520 00002000:00010000:1.0:1788572542.898987:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x49:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:1.0:1788572542.898989:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x49:0x0] updating lvb ctime from disk: 1788572542 -> 1788572524 00002000:00010000:1.0:1788572542.898991:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x49:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572542.899074:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360375168:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 366us (583us total) trans 4294967588 rc 0/0 00000100:00100000:1.0:1788572542.899082:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1107 00000100:00100000:3.0:1788572542.903467:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.903524:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360375424 00000100:00100000:1.0:1788572542.903534:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360375424 00000100:00100000:1.0:1788572542.903596:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4149 00000100:00100000:1.0:1788572542.903606:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35911:x1875453360375424:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.903681:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.903699:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.903709:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.903744:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.903759:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 65 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.903773:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382356352:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.903866:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.910885:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360374656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 71909us (72070us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.910899:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1105 00000001:00080000:0.0:1788572542.910956:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967588 is committed 00000001:00080000:0.0:1788572542.910968:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967585 is committed 00000100:00100000:1.0:1788572542.912220:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.913045:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382356352:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572542.913078:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.913233:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360375552 00000100:00100000:1.0:1788572542.913240:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360375552 00010000:00010000:1.0:1788572542.913284:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.913351:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 982 00000100:00100000:1.0:1788572542.913362:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360375552:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.913378:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572542.913385:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913399:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913421:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1628d2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x84:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb0b expref: 66 pid: 5839 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913443:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572542.913448:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913469:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913477:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913502:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572542.913555:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360375552:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 195us (1356us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.913564:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 982 00010000:00010000:1.0:1788572542.913588:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913595:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913620:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913626:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913633:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913638:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572542.913643:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913650:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913658:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572542.913661:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572542.913664:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.913670:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb162965 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.913706:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35911:x1875453360375424:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10102us (10258us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572542.913714:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4149 00000100:00100000:1.0:1788572542.917646:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.917757:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360375680 00000100:00100000:0.0:1788572542.917767:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360375680 00000100:00100000:0.0:1788572542.917826:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1108 00000100:00100000:0.0:1788572542.917836:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360375680:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.917881:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.917901:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.917910:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572542.917914:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.917919:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 4/1,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.917971:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.917987:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 13 pid: 6681 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.918032:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382356480:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572542.918112:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.924660:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.924732:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.924759:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360375936 00000100:00100000:3.0:1788572542.924769:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360375936 00000100:00100000:1.0:1788572542.924801:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382356480:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572542.924822:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4150 00010000:00010000:1.0:1788572542.924832:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.924832:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35910:x1875453360375936:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572542.924892:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572542.924921:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162934 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb43 expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.924947:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572542.924953:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572542.924987:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.925000:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.925043:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572542.925220:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d534ed56 reply state 0000000022f2776e transno 4294970409 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.925228:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572542.927630:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360376064 00000100:00100000:1.0:1788572542.927646:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360376064 00010000:00010000:1.0:1788572542.927693:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 14 pid: 6681 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.927775:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 983 00000100:00100000:1.0:1788572542.927785:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360376064:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572542.927800:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1788572542.927810:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35910:x1875453360375936:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2979us (3160us total) trans 4294970409 rc 0/0 00000100:00100000:3.0:1788572542.927820:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4150 00010000:00010000:1.0:1788572542.927823:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 14 pid: 6681 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.927836:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 14 pid: 6681 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.927868:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000040a9311b/0x85ace67feb162950 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x49:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb5f expref: 14 pid: 6681 timeout: 990 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.927884:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.927889:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.927895:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.927912:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.928023:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572542.929902:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.929920:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.929954:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.929962:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.929972:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.929980:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572542.929985:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.929999:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.930034:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572542.930042:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16296c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.930128:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360375680:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 12290us (12488us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.930138:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1108 00000100:00100000:1.0:1788572542.930880:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360376064:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3091us (6142us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.930891:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 983 00000100:00100000:3.0:1788572542.935946:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.936045:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572542.936176:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.936190:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.936197:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.936202:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572542.936208:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.936216:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.936236:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572542.936240:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572542.936246:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.936254:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162973 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.936365:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360376448 00000100:00100000:1.0:1788572542.936377:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360376448 00000100:00100000:1.0:1788572542.936446:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4151 00000100:00100000:1.0:1788572542.936467:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35911:x1875453360376448:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.936565:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936579:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936591:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936872:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936884:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936892:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936938:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.936953:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 65 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.936967:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382356608:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.937081:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.937563:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360376192 00000100:00100000:0.0:1788572542.937570:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360376192 00000100:00100000:0.0:1788572542.937610:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1109 00000100:00100000:0.0:1788572542.937617:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360376192:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572542.937805:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:0.0:1788572542.937809:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb mtime from disk: 1788572542 -> 1788572519 00002000:00010000:0.0:1788572542.937812:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb atime from disk: 0 -> 1788572519 00002000:00010000:0.0:1788572542.937814:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb ctime from disk: 1788572542 -> 1788572525 00002000:00010000:0.0:1788572542.937817:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4a:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572542.937851:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360376192:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 234us (1919us total) trans 4294967589 rc 0/0 00000100:00100000:0.0:1788572542.937859:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1109 00000100:00100000:0.0:1788572542.945326:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.945432:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.945471:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360376576 00000100:00100000:0.0:1788572542.945480:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360376576 00010000:00010000:0.0:1788572542.945523:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 66 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.945563:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 984 00000100:00100000:0.0:1788572542.945572:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360376576:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.945588:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.945595:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 66 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.945609:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 66 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.945638:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572542.945644:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.945655:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.945655:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360376832 00000100:00100000:2.0:1788572542.945660:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360376832 00010000:00010000:0.0:1788572542.945663:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.945681:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572542.945699:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4152 00000100:00100000:2.0:1788572542.945706:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35910:x1875453360376832:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00000100:00100000:0.0:1788572542.945730:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360376576:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 161us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.945737:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 984 00010000:00010000:2.0:1788572542.945763:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.945777:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.945785:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.945815:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.945828:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 66 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.945843:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30df80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382356736:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.945899:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c123c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572542.949079:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1628fc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb20 expref: 66 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.949106:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382356608:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.949131:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.949142:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.949148:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572542.949595:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970410 00000001:00080000:1.0:1788572542.949771:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970411 00000001:00080000:1.0:1788572542.949822:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970412 00000100:00100000:1.0:1788572542.949913:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382356864:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572542.952165:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572542.952234:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9600 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.953684:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382356864 00000100:00100000:3.0:1788572542.953703:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382356864 00000100:00100000:3.0:1788572542.953824:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1788572542.953837:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:5838:x1875453382356864:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572542.954021:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:5838:x1875453382356864:12345-0@lo:1000:mdt00_001.0 Request processed in 187us (1863us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572542.954029:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:1.0:1788572542.954609:0:5838:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967580 gen 2 00000004:00080000:1.0:1788572542.954617:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967580 committed transno 4294967580 00000100:00100000:1.0:1788572542.954641:0:5838:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627f5b67800 x1875453382355840/t4294967585(4294967585) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1788572558 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572542.954654:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382356864:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1788572542.955215:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970413 00000100:00100000:1.0:1788572542.955319:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382356992:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572542.955531:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572542.955560:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c978d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572542.955642:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382356992 00000100:00100000:0.0:1788572542.955649:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382356992 00000100:00100000:0.0:1788572542.955699:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:0.0:1788572542.955709:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:5838:x1875453382356992:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572542.956522:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:5838:x1875453382356992:12345-0@lo:1000:mdt00_001.0 Request processed in 812us (988us total) trans 4294967589 rc 0/0 00000100:00100000:0.0:1788572542.956531:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:0.0:1788572542.956596:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382356992:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1788572542.956633:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956643:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956656:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956662:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.956667:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956675:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956692:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572542.956696:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572542.956701:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956707:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb162981 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x84:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956742:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956749:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956755:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956760:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.956763:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956774:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956782:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572542.956784:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572542.956787:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956792:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb15f3e9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x83:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956814:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956826:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956835:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.956841:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957165:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957171:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957176:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.957179:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957186:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957194:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572542.957196:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.957433:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957442:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16298f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x83:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957467:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957473:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957479:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957484:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bad600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.957488:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957494:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957501:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572542.957504:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572542.957507:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.957517:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb16297a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x83:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.958001:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35911:x1875453360376448:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 21542us (21952us total) trans 4294970414 rc 0/0 00000100:00100000:0.0:1788572542.958026:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4151 00000020:00080000:0.0:1788572542.958030:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572542.958115:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360374784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 117699us (118074us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.958122:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1106 00000001:00080000:0.0:1788572542.958145:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967587 is committed 00000100:00100000:1.0:1788572542.959735:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.959837:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360376960 00000100:00100000:0.0:1788572542.959845:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360376960 00010000:00010000:0.0:1788572542.959885:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 69 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.959940:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 985 00000100:00100000:0.0:1788572542.959950:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2349:x1875453360376960:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.959965:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.959972:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 69 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.959984:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 69 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.960030:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572542.960037:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.960046:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.960054:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.960074:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.960115:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2349:x1875453360376960:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 167us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.960123:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 985 00000100:00100000:3.0:1788572542.964574:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.964618:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.964683:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360377216 00000100:00100000:0.0:1788572542.964693:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360377216 00000100:00100000:0.0:1788572542.964754:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4153 00000100:00100000:0.0:1788572542.964765:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35912:x1875453360377216:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.964829:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.964867:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162911 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb2e expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.964885:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572542.964890:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572542.964921:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.965067:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.965077:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572542.965488:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d018117d reply state 0000000057849af0 transno 4294970415 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.965498:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572542.965603:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35912:x1875453360377216:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 838us (1037us total) trans 4294970415 rc 0/0 00000100:00100000:0.0:1788572542.965611:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4153 00000100:00100000:1.0:1788572542.965682:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360377472 00000100:00100000:1.0:1788572542.965688:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360377472 00000100:00100000:1.0:1788572542.965726:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 899 00000100:00100000:1.0:1788572542.965734:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35911:x1875453360377472:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572542.967783:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x84:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572542.967910:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970416 00010000:00010000:2.0:1788572542.969087:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb1628e7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x85:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb19 expref: 72 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.969112:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30df80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382356736:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.969140:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970181:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970200:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970222:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970230:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970238:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970245:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.970267:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572542.970616:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35911:x1875453360377472:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 4878us (5993us total) trans 4294970417 rc 0/0 00000100:00100000:1.0:1788572542.970628:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 899 00000020:00080000:1.0:1788572542.970635:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572542.970773:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970782:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970808:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970813:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970819:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970824:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.970828:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970835:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970862:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572542.970866:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572542.970870:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.970875:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162988 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.970913:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35910:x1875453360376832:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 25207us (25598us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572542.970932:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4152 00010000:00010000:0.0:1788572542.971417:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.971426:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.971432:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.971437:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.971440:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.971445:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.971465:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572542.971468:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572542.971472:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.971477:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162996 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.971695:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572542.971703:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.971711:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7b90c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572542.971714:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.971721:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.971731:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x84:0x0].0x0 (ffff9627ffc16900) 00010000:00010000:2.0:1788572542.971733:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572542.971736:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 1/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.971741:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb16299d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x84:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.971758:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572542.971760:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572542.971762:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572542.975270:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.975528:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360377600 00000100:00100000:0.0:1788572542.975539:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360377600 00000100:00100000:0.0:1788572542.975592:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1110 00000100:00100000:0.0:1788572542.975604:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360377600:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572542.975854:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:0.0:1788572542.975858:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb mtime from disk: 1788572542 -> 1788572520 00002000:00010000:0.0:1788572542.975861:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:0.0:1788572542.975863:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb ctime from disk: 1788572542 -> 1788572525 00002000:00010000:0.0:1788572542.975866:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4a:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:0.0:1788572542.975944:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360377600:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 342us (684us total) trans 4294967590 rc 0/0 00000100:00100000:0.0:1788572542.975952:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1110 00000100:00100000:3.0:1788572542.980827:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572542.980914:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360377728 00000100:00100000:1.0:1788572542.980933:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360377728 00000100:00100000:1.0:1788572542.981052:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1111 00000100:00100000:1.0:1788572542.981063:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360377728:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572542.981111:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.981131:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.981139:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788572542.981143:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572542.981148:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.981185:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.981215:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 11 pid: 10416 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.981232:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382357120:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572542.981609:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9600 going to sleep for 11 seconds 00000100:00100000:3.0:1788572542.983691:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572542.983790:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360377984 00000100:00100000:2.0:1788572542.983800:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360377984 00000100:00100000:2.0:1788572542.983863:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4154 00000100:00100000:2.0:1788572542.983874:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35912:x1875453360377984:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572542.983960:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.983981:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.983992:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 4/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.984069:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572542.984083:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 69 pid: 23058 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572542.984098:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30df80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382357248:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572542.984174:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c123c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572542.986978:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30df80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382357248:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572542.987042:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.987654:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382357120:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572542.987674:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572542.991113:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572542.991148:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.991324:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360378112 00000100:00100000:0.0:1788572542.991333:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360378112 00010000:00010000:0.0:1788572542.991377:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 70 pid: 23058 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.991420:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360378240 00000100:00100000:0.0:1788572542.991423:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360378240 00010000:00010000:0.0:1788572542.991437:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 12 pid: 10416 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.991469:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 986 00000100:00100000:0.0:1788572542.991478:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360378112:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.991492:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.991499:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 70 pid: 23058 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991511:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 70 pid: 23058 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991532:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb16288c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x86:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efacc expref: 70 pid: 23058 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991548:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572542.991553:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991564:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991571:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991596:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.991637:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2350:x1875453360378112:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 161us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.991644:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 986 00000100:00100000:0.0:1788572542.991658:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 987 00000100:00100000:0.0:1788572542.991664:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e50450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360378240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572542.991672:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572542.991694:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 12 pid: 10416 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991712:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 12 pid: 10416 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991729:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c36dd831/0x85ace67feb162957 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb66 expref: 12 pid: 10416 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991745:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.991749:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572542.991754:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991764:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991778:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572542.991793:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e50450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360378240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 129us (644us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572542.991799:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 987 00010000:00010000:0.0:1788572542.991846:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991854:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991882:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991893:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991899:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991904:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572542.991909:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991916:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991937:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572542.991939:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572542.991943:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 2/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.991949:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ab lrc: 0/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.991983:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35912:x1875453360377984:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8111us (8302us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572542.991991:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4154 00010000:00010000:1.0:1788572542.992551:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992563:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992587:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992594:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992600:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992611:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572542.992615:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992622:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992633:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572542.992641:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1629a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572542.992694:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360377728:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11633us (11877us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572542.992701:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1111 00000100:00100000:3.0:1788572542.996708:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572542.996797:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360378496 00000100:00100000:0.0:1788572542.996810:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360378496 00000100:00100000:0.0:1788572542.996897:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4155 00000100:00100000:0.0:1788572542.996910:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35910:x1875453360378496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572542.997064:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997084:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997097:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997642:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997656:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 69 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997664:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997700:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 69 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572542.997714:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 69 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572542.997730:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382357376:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572542.997857:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.001553:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.001643:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360378624 00000100:00100000:3.0:1788572543.001655:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360378624 00000100:00100000:3.0:1788572543.001720:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1112 00000100:00100000:3.0:1788572543.001731:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360378624:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.001788:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.001809:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.001819:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788572543.001823:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.001828:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.001872:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.001886:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 12 pid: 20911 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.001902:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382357504:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.001995:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627cbda5000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.005526:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382357504:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572543.005563:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.008325:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.008484:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360378752 00000100:00100000:1.0:1788572543.008494:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360378752 00010000:00010000:1.0:1788572543.008556:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 70 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.008604:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 988 00000100:00100000:1.0:1788572543.008620:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360378752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572543.008634:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.008642:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 70 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.008655:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 70 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.008677:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:1.0:1788572543.008682:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.008692:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.008700:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.008721:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572543.008779:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360378752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 166us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.008785:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 988 00000100:00080000:2.0:1788572543.009475:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1788572543.009683:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162903 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb27 expref: 69 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.009699:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382357376:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.009720:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.009730:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.009735:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572543.009920:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970418 00000001:00080000:0.0:1788572543.010286:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970419 00000001:00080000:0.0:1788572543.010507:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970420 00000100:00100000:0.0:1788572543.010603:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382357632:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572543.010667:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.010707:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.010757:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382357632 00000100:00100000:2.0:1788572543.010765:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382357632 00000100:00100000:2.0:1788572543.010814:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:2.0:1788572543.010823:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382357632:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1788572543.010901:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382357632:12345-0@lo:1000:mdt00_001.0 Request processed in 79us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.010909:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1788572543.011140:0:5838:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967588 gen 2 00000004:00080000:0.0:1788572543.011146:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967582 committed transno 4294967588 00000004:00080000:0.0:1788572543.011164:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967583 committed transno 4294967588 00000004:00080000:0.0:1788572543.011177:0:5838:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b64700, transno 4294967586 00000004:00080000:0.0:1788572543.011189:0:5838:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b67b80, transno 4294967587 00000004:00080000:0.0:1788572543.011192:0:5838:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b66680, transno 4294967588 00000100:00100000:0.0:1788572543.011197:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382357632:0@lo:1000:mdt00_001.0 00000004:00080000:2.0:1788572543.011594:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 3 done 00000001:00080000:0.0:1788572543.012472:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970421 00000100:00100000:0.0:1788572543.012540:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382357760:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572543.012586:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.012670:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.012730:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382357760 00000100:00100000:3.0:1788572543.012736:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382357760 00000100:00100000:3.0:1788572543.012790:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1788572543.012798:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382357760:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572543.013721:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.013813:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360378880 00000100:00100000:0.0:1788572543.013818:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360378880 00010000:00010000:0.0:1788572543.013847:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 13 pid: 20911 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.013877:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 989 00000100:00100000:0.0:1788572543.013884:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360378880:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.013896:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.013916:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 13 pid: 20911 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.013939:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 13 pid: 20911 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.013962:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb162949 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb58 expref: 13 pid: 20911 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.013979:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572543.013983:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572543.013989:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.013998:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014047:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.014088:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360378880:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 204us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.014095:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 989 00000100:00100000:3.0:1788572543.014379:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382357760:12345-0@lo:1000:mdt00_001.0 Request processed in 1581us (1791us total) trans 4294967590 rc 0/0 00000100:00100000:3.0:1788572543.014388:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00010000:00010000:3.0:1788572543.014411:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014420:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014443:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014462:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014471:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014484:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572543.014489:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014498:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014510:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.014518:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb1629c0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.014558:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360378624:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 12829us (13016us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.014564:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1112 00000100:00100000:0.0:1788572543.014691:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382357760:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1788572543.014727:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014735:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014741:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014745:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.014749:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014755:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014765:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572543.014768:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572543.014772:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014778:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629b9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x85:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014810:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014817:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014822:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014827:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.014830:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014835:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014841:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572543.014843:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572543.014846:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014851:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb15f380 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014870:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014882:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014890:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014896:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014901:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014906:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014911:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.014914:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014918:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.014988:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572543.014991:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.014994:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015000:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629c7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x89:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015617:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015625:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015631:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015636:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.015639:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015644:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015657:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572543.015659:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572543.015663:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.015669:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1629b2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.015825:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35910:x1875453360378496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 18916us (19125us total) trans 4294970422 rc 0/0 00000100:00100000:0.0:1788572543.015834:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4155 00000020:00080000:0.0:1788572543.015838:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572543.018869:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.019106:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360379136 00000100:00100000:3.0:1788572543.019117:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360379136 00000100:00100000:3.0:1788572543.019174:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4156 00000100:00100000:3.0:1788572543.019183:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35912:x1875453360379136:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.019285:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 3/0,1 mode: --/PW res: [0x200000401:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019437:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019464:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019718:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019730:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 73 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019740:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019773:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 73 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.019788:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 73 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.019804:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382357888:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.019890:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.023274:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382357888:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572543.023320:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.024485:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.024723:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360379264 00000100:00100000:2.0:1788572543.024734:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360379264 00010000:00010000:2.0:1788572543.024781:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 74 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.024815:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 990 00000100:00100000:2.0:1788572543.024824:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2350:x1875453360379264:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572543.024843:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572543.024851:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 74 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.024864:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 74 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.024889:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1628d9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb12 expref: 74 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.024908:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:2.0:1788572543.024914:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.024935:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.024944:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.024975:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572543.025062:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2350:x1875453360379264:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 238us (582us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.025070:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 990 00010000:00010000:3.0:1788572543.025584:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.025597:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572543.025807:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970423 00000001:00080000:3.0:1788572543.026076:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970424 00000001:00080000:3.0:1788572543.026125:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970425 00000100:00100000:3.0:1788572543.026585:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382358016:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572543.026650:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572543.026721:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.028913:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382358016 00000100:00100000:1.0:1788572543.028940:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382358016 00000100:00100000:1.0:1788572543.029109:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:1.0:1788572543.029121:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382358016:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1788572543.029272:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382358016:12345-0@lo:1000:mdt00_005.0 Request processed in 153us (2621us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.029429:0:23058:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967587 gen 2 00000004:00080000:0.0:1788572543.029437:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b67800, transno 4294967585 00000004:00080000:0.0:1788572543.029449:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b64a80, transno 4294967586 00000004:00080000:0.0:1788572543.029463:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b66300, transno 4294967587 00000100:00100000:0.0:1788572543.029470:0:23058:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff962801f92d80 x1875453382356992/t4294967589(4294967589) o1000->lustre-OST0001-osc-MDT0000@0@lo:24/4 lens 408/4320 e 0 to 0 dl 1788572558 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mdt00_001.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572543.029483:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382358016:0@lo:1000:mdt00_005.0 00000001:00080000:0.0:1788572543.029900:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970426 00000100:00100000:0.0:1788572543.029996:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382358144:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572543.030070:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.030096:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.030115:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382358144 00000100:00100000:0.0:1788572543.030119:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382358144 00000100:00100000:2.0:1788572543.030240:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.030267:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360379520 00000100:00100000:2.0:1788572543.030271:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360379520 00000100:00100000:2.0:1788572543.030513:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 900 00000100:00100000:2.0:1788572543.030524:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd1c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35910:x1875453360379520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572543.030840:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x85:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572543.030886:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970428 00010000:00010000:2.0:1788572543.030998:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.031040:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.031053:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.031063:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.031069:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.031076:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.031094:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572543.031179:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd1c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35910:x1875453360379520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 656us (939us total) trans 4294970429 rc 0/0 00000100:00100000:2.0:1788572543.031189:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 900 00000020:00080000:2.0:1788572543.031194:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572543.032495:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572543.032508:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.032520:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572543.032524:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.032534:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.032551:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x85:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:2.0:1788572543.032554:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572543.032560:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 1/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.032566:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629dc lrc: 0/0,0 mode: --/PW res: [0x200000402:0x85:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.032590:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572543.032592:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572543.032593:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572543.032812:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000004:00080000:1.0:1788572543.033451:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:0.0:1788572543.033639:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:0.0:1788572543.033655:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382358144:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572543.033897:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:23058:x1875453382358144:12345-0@lo:1000:mdt00_005.0 Request processed in 248us (3826us total) trans 4294967591 rc 0/0 00000100:00100000:0.0:1788572543.033906:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1788572543.034048:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382358144:0@lo:1000:mdt00_005.0 00010000:00010000:1.0:1788572543.034097:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034106:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034113:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034119:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.034124:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034131:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034145:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572543.034148:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572543.034153:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034158:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1629d5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x86:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034192:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034197:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034220:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034224:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800934600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.034227:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034232:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034239:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:1.0:1788572543.034242:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572543.034245:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034251:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800934600/0x85ace67feb15f421 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034275:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034547:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034560:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034568:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034573:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034580:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034585:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.034588:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034593:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034602:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:1.0:1788572543.034604:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.034607:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034613:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1629e3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x88:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034623:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034629:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034635:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034640:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.034643:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034661:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034669:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:1.0:1788572543.034672:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572543.034674:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 2/0,0 mode: --/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.034681:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb1629ce lrc: 0/0,0 mode: --/PW res: [0x200000401:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.034846:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35912:x1875453360379136:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 15660us (15984us total) trans 4294970427 rc 0/0 00000100:00100000:1.0:1788572543.034855:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4156 00000020:00080000:1.0:1788572543.034860:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:2.0:1788572543.035065:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572543.035071:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572543.035112:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 139 count 101 00000040:00080000:2.0:1788572543.035119:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:2.0:1788572543.035257:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30c380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382358272:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572543.035326:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c63cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.043827:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.043872:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.043898:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.043925:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.043954:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360380032 00000100:00100000:2.0:1788572543.043964:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360380032 00000100:00100000:2.0:1788572543.044039:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 901 00000100:00100000:2.0:1788572543.044049:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35912:x1875453360380032:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.044399:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360379648 00000100:00100000:0.0:1788572543.044404:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360379648 00000100:00100000:0.0:1788572543.044430:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360379776 00000100:00100000:0.0:1788572543.044432:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360379776 00000100:00100000:0.0:1788572543.044444:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360380160 00000100:00100000:0.0:1788572543.044446:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360380160 00000100:00100000:0.0:1788572543.044474:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4157 00000100:00100000:0.0:1788572543.044482:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35918:x1875453360379648:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572543.044503:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb4:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572543.044606:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35918:x1875453360379648:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 125us (791us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.044613:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4157 00000100:00100000:0.0:1788572543.044629:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4158 00000100:00100000:0.0:1788572543.044634:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35919:x1875453360379776:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572543.044645:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb5:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572543.044680:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35919:x1875453360379776:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 46us (808us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.044686:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4158 00000100:00100000:0.0:1788572543.044697:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4159 00000100:00100000:0.0:1788572543.044702:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35917:x1875453360380160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572543.044714:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x82:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572543.044750:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35917:x1875453360380160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 47us (824us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.044756:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4159 00000004:00080000:2.0:1788572543.045055:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x86:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572543.045093:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970430 00010000:00010000:2.0:1788572543.045135:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.045150:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.045161:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.045169:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.045175:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.045181:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.045449:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572543.045535:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35912:x1875453360380032:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1486us (1634us total) trans 4294970431 rc 0/0 00000100:00100000:2.0:1788572543.045543:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 901 00000020:00080000:2.0:1788572543.045547:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572543.049155:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572543.049173:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.049184:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572543.049189:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.049201:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.049231:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x86:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:0.0:1788572543.049234:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572543.049239:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 1/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.049246:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1629ea lrc: 0/0,0 mode: --/PW res: [0x200000402:0x86:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.049273:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.049275:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.049277:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572543.049971:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.050860:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360380288 00000100:00100000:0.0:1788572543.050871:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360380288 00000100:00100000:0.0:1788572543.050941:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4160 00000100:00100000:0.0:1788572543.050951:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35918:x1875453360380288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.051059:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051081:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051092:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051106:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051112:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051117:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051143:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051149:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051154:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.051192:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb4:0x0]: returning size 3 00000004:00200000:0.0:1788572543.051195:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.051681:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051696:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051708:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051715:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051721:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.051725:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051731:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051742:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572543.051745:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.051750:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051761:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1629f1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051779:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051785:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051791:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051795:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.051799:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051805:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051813:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x5f50981c (ffff9627e7451b00) 00010000:00010000:0.0:1788572543.051816:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.051819:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051824:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1629f8 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x5f50981c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051844:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051851:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051857:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051861:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.051864:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051870:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051877:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:0.0:1788572543.051879:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.051883:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.051888:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1629ff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.051993:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35918:x1875453360380288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1042us (2026us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.052030:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4160 00000100:00100000:3.0:1788572543.053277:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30c380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382358272:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572543.054063:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.054262:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360380416 00000100:00100000:0.0:1788572543.054270:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360380416 00000100:00100000:0.0:1788572543.054334:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4161 00000100:00100000:0.0:1788572543.054344:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35919:x1875453360380416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.054423:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054442:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054462:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054479:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054487:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054494:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054521:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054530:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054537:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.054577:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb5:0x0]: returning size 3 00000004:00200000:0.0:1788572543.054582:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.054594:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054604:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054611:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054618:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054624:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.054628:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054635:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054645:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572543.054648:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.054653:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054662:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a06 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054679:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054685:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054691:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054696:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.054699:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054705:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054712:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x1d12f327 (ffff9627e7451a00) 00010000:00010000:0.0:1788572543.054714:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.054716:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054722:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a0d lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1d12f327 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054736:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054742:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054748:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054752:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.054754:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054760:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054765:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:0.0:1788572543.054768:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.054770:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.054776:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.054862:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35919:x1875453360380416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 518us (802us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.054870:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4161 00000100:00100000:3.0:1788572543.058491:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.059396:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360380544 00000100:00100000:1.0:1788572543.059409:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360380544 00000100:00100000:1.0:1788572543.059481:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4162 00000100:00100000:1.0:1788572543.059491:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35917:x1875453360380544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.059579:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059599:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059612:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059645:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059652:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059658:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059685:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059693:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059700:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572543.059737:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x82:0x0]: returning size 3 00000004:00200000:1.0:1788572543.059741:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572543.059755:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059767:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059773:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059781:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059792:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.059797:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059805:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059818:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572543.059821:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.059826:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059833:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a1b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059844:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059860:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059866:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059872:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.059874:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059879:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059890:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3b60c571 (ffff9627ffc16c00) 00010000:00010000:1.0:1788572543.059893:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.059895:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059901:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a22 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3b60c571 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059915:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059921:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059933:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059938:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.059941:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059948:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059956:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:1.0:1788572543.059958:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.059961:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.059966:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a29 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.060083:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35917:x1875453360380544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 591us (1600us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.060090:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4162 00000100:00100000:2.0:1788572543.060315:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.060480:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360380800 00000100:00100000:0.0:1788572543.060486:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360380800 00000100:00100000:0.0:1788572543.060527:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4163 00000100:00100000:0.0:1788572543.060536:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35919:x1875453360380800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.060548:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.060564:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162a30 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb6d expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060645:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060657:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060666:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.060676:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb5:0x0]: returning size 3 00010000:00010000:0.0:1788572543.060799:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060806:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060812:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060817:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.060820:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060826:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060835:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:0.0:1788572543.060837:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.060841:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060850:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162a37 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060891:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162a30 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb6d expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060901:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572543.060903:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572543.060907:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162a30 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb6d expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.060916:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572543.060964:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35919:x1875453360380800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 429us (649us total) trans 4294970432 rc 301/301 00000100:00100000:0.0:1788572543.060971:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4163 00000100:00100000:2.0:1788572543.064389:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.064558:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381056 00000100:00100000:0.0:1788572543.064568:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381056 00000100:00100000:0.0:1788572543.064634:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4164 00000100:00100000:0.0:1788572543.064644:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35918:x1875453360381056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.064667:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.064687:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a45 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb74 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064806:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064822:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064833:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.064844:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb4:0x0]: returning size 3 00010000:00010000:0.0:1788572543.064951:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064959:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064966:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064972:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.064977:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064984:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.064998:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:0.0:1788572543.065044:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.065050:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.065058:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a4c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.065108:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a45 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb74 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.065124:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572543.065126:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572543.065131:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a45 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb74 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.065145:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:0.0:1788572543.065214:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35918:x1875453360381056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 572us (838us total) trans 4294970433 rc 301/301 00000100:00100000:0.0:1788572543.065222:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4164 00000100:00100000:1.0:1788572543.070061:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.070097:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.070197:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381184 00000100:00100000:0.0:1788572543.070218:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381184 00000100:00100000:0.0:1788572543.070268:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381440 00000100:00100000:0.0:1788572543.070271:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381440 00000100:00100000:0.0:1788572543.070317:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4165 00000100:00100000:0.0:1788572543.070328:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35918:x1875453360381184:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572543.070368:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb4:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572543.070492:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35918:x1875453360381184:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 165us (487us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.070500:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4165 00000100:00100000:0.0:1788572543.070520:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4166 00000100:00100000:0.0:1788572543.070527:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35917:x1875453360381440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.070538:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.070561:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a5a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb7b expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070659:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070676:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070688:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.070699:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x82:0x0]: returning size 3 00010000:00010000:0.0:1788572543.070807:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070815:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070822:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070827:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.070832:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070840:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070852:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:0.0:1788572543.070855:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.070861:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070871:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a61 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070924:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a5a lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb7b expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070935:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572543.070937:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572543.070941:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a5a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb7b expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.070951:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572543.070983:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35917:x1875453360381440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 457us (885us total) trans 4294970434 rc 301/301 00000100:00100000:0.0:1788572543.070989:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4166 00000100:00100000:0.0:1788572543.075511:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.075613:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381568 00000100:00100000:2.0:1788572543.075623:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381568 00000100:00100000:2.0:1788572543.075686:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4167 00000100:00100000:2.0:1788572543.075697:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35919:x1875453360381568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572543.075736:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb5:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572543.075859:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35919:x1875453360381568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 164us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.075867:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4167 00000100:00100000:3.0:1788572543.079429:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.079602:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381696 00000100:00100000:0.0:1788572543.079612:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381696 00000100:00100000:0.0:1788572543.079672:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4168 00000100:00100000:0.0:1788572543.079683:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35917:x1875453360381696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572543.079717:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x82:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572543.079836:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35917:x1875453360381696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 155us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.079843:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4168 00000100:00100000:3.0:1788572543.082561:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.082660:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381824 00000100:00100000:1.0:1788572543.082673:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381824 00000100:00100000:1.0:1788572543.082746:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4169 00000100:00100000:1.0:1788572543.082760:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35918:x1875453360381824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.082791:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.082818:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a6f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb82 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.082905:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.082937:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.082951:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.082969:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.082976:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.082983:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084180:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a84 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084198:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a84 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084218:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a84 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.084254:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:0.0:1788572543.084258:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.084280:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a84 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084487:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084496:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084503:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084509:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.084514:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084522:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084535:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572543.084538:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.084542:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084552:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162a76 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084565:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084571:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084577:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084581:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.084584:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084590:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084597:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc16c00) 00010000:00010000:0.0:1788572543.084600:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.084603:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084609:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162a7d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084650:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162a6f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb82 expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084669:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a84 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb82 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.084677:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572543.084679:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.084681:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572543.084758:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35918:x1875453360381824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2001us (2208us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.084767:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4169 00000100:00100000:2.0:1788572543.084901:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.085985:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360381952 00000100:00100000:0.0:1788572543.085997:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360381952 00000100:00100000:0.0:1788572543.086072:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4170 00000100:00100000:0.0:1788572543.086082:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35917:x1875453360381952:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.086101:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.086117:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a8b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb89 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086185:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086201:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086226:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086245:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086254:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086260:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086754:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162aa0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086764:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162aa0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086771:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162aa0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.086797:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572543.086800:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.086821:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162aa0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086831:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086838:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086844:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086906:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.086911:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086919:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086929:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572543.086932:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.086936:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086942:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162a92 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086957:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086961:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086966:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086971:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.086973:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086978:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086984:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627e7451a00) 00010000:00010000:0.0:1788572543.086986:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.086989:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.086997:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162a99 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.087063:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162a8b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb89 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.087076:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162aa0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb89 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.087082:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572543.087084:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.087087:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572543.087265:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35917:x1875453360381952:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1183us (2363us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.087274:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4170 00000100:00100000:3.0:1788572543.091369:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.091436:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.091596:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360382208 00000100:00100000:0.0:1788572543.091606:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360382208 00000100:00100000:0.0:1788572543.091690:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360382336 00000100:00100000:0.0:1788572543.091693:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360382336 00000100:00100000:1.0:1788572543.091725:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4171 00000100:00100000:1.0:1788572543.091737:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35918:x1875453360382208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.091760:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.091763:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572543.091788:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162a84 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb82 expref: 86 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.091811:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.091816:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.091830:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162aa7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb90 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572543.092498:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4b:0x0], index 0: 0 00010000:00010000:1.0:1788572543.092695:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162aae lrc: 3/1,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092709:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162aae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092719:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162aae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092779:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162aa7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb90 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092795:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162aae lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb90 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092809:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.092812:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572543.092814:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3714000, rc 0) 00000100:00100000:1.0:1788572543.092885:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35918:x1875453360382208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1149us (1525us total) trans 4294970435 rc 0/0 00000100:00100000:1.0:1788572543.092892:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4171 00000100:00100000:1.0:1788572543.092920:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4172 00000100:00100000:1.0:1788572543.092928:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35919:x1875453360382336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.092938:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.092947:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162abc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb97 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092989:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.092999:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.093029:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.093041:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.093048:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.093055:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.093271:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162ad1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.093281:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162ad1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094439:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162ad1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572543.094489:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:1.0:1788572543.094494:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572543.094512:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162ad1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094525:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094531:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094539:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094544:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.094549:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094556:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094570:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572543.094574:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.094578:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094584:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162ac3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094600:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094606:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094612:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094616:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.094618:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094623:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094630:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc16b00) 00010000:00010000:1.0:1788572543.094632:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.094636:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094641:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162aca lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094679:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162abc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb97 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094694:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162ad1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb97 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.094701:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.094703:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572543.094705:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679800, rc 0) 00000100:00100000:1.0:1788572543.094809:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35919:x1875453360382336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1880us (3371us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.094816:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4172 00010000:00010000:3.0:1788572543.098984:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572543.101573:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.101822:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360382848 00000100:00100000:1.0:1788572543.101832:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360382848 00000100:00100000:1.0:1788572543.101898:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4173 00000100:00100000:1.0:1788572543.101908:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35918:x1875453360382848:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.101977:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572543.102033:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162aae lrc: 0/0,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb90 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.102052:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.102069:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162ad1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb97 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.102077:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.102080:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.102102:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.102120:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 167 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.102129:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.102161:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 167 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.102174:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 167 pid: 18263 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.102188:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382358400:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572543.102490:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c93d9600 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.105037:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.105133:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360382592 00000100:00100000:0.0:1788572543.105143:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360382592 00000100:00100000:0.0:1788572543.105200:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4174 00000100:00100000:0.0:1788572543.105219:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35917:x1875453360382592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.105240:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.105243:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572543.105272:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162aa0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb89 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.105308:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572543.105314:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572543.105326:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162adf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efb9e expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572543.105831:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4b:0x0], index 1: 0 00010000:00010000:0.0:1788572543.106045:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162ae6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.106060:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.106069:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.106120:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162adf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb9e expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.106139:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162ae6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb9e expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.106146:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:0.0:1788572543.106149:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572543.106151:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572543.106220:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35917:x1875453360382592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1003us (1221us total) trans 4294970436 rc 0/0 00000100:00100000:0.0:1788572543.106227:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4174 00000100:00100000:3.0:1788572543.110349:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.110500:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360382976 00000100:00100000:0.0:1788572543.110510:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360382976 00010000:00010000:0.0:1788572543.110559:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 168 pid: 18263 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.110607:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 991 00000100:00100000:0.0:1788572543.110618:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:2349:x1875453360382976:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.110632:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.110639:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 168 pid: 18263 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.110653:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 168 pid: 18263 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.110678:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572543.110683:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.110693:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.110702:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.110724:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.110773:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:2349:x1875453360382976:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 157us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.110781:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 991 00010000:00010000:1.0:1788572543.110881:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb15da4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddfa expref: 167 pid: 18263 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.110899:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382358400:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.110921:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.110943:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.110948:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.110971:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.110981:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.110988:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111254:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111263:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111269:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111275:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.111279:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111287:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111438:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.111441:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.111446:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111461:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb162ad8 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111473:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111481:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111486:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111491:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.111494:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111500:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111508:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x9c69e50d (ffff9627ffc16b00) 00010000:00010000:1.0:1788572543.111511:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.111513:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.111520:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162af4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x9c69e50d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.111588:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35918:x1875453360382848:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 9680us (10024us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572543.111596:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4173 00000100:00080000:2.0:1788572543.111811:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788572543.113345:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.113476:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360383104 00000100:00100000:0.0:1788572543.113488:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360383104 00000100:00100000:0.0:1788572543.113541:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4175 00000100:00100000:0.0:1788572543.113552:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35917:x1875453360383104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.113574:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.113593:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162afb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efba5 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.113657:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.113672:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.113682:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.113698:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.113705:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.113712:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114082:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162b10 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114093:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162b10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114101:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162b10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.114129:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572543.114133:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.114154:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162b10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114165:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114172:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114179:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114184:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.114188:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114196:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114207:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572543.114210:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.114214:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114219:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162b02 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114233:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114243:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114249:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114254:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.114256:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114262:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114269:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xd9410b4a (ffff9627e7451b00) 00010000:00010000:0.0:1788572543.114272:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.114275:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114280:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162b09 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xd9410b4a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114562:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162afb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efba5 expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114579:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162b10 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efba5 expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.114586:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572543.114589:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.114592:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:0.0:1788572543.114656:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35917:x1875453360383104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1106us (1318us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.114664:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4175 00000100:00100000:2.0:1788572543.118850:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.118940:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360383232 00000100:00100000:0.0:1788572543.118952:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360383232 00000100:00100000:0.0:1788572543.119052:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4176 00000100:00100000:0.0:1788572543.119064:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35918:x1875453360383232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.119084:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.119111:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162b17 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbac expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.119152:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.119165:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.119177:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.119845:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162b17 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.119880:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.119889:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:0.0:1788572543.119892:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572543.119898:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572543.120045:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35918:x1875453360383232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 982us (1202us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.120053:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4176 00000100:00100000:3.0:1788572543.122858:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.123040:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360383488 00000100:00100000:0.0:1788572543.123052:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360383488 00000100:00100000:0.0:1788572543.123127:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4177 00000100:00100000:0.0:1788572543.123138:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35919:x1875453360383488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.123164:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.123185:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162b25 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbb3 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572543.123707:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4c:0x0], index 0: 0 00010000:00010000:0.0:1788572543.127905:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162b2c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.127988:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162b2c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.128016:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162b2c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.128129:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162b25 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbb3 expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.128155:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162b2c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbb3 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.128165:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7451100) 00010000:00010000:0.0:1788572543.128168:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572543.128172:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:0.0:1788572543.128262:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35919:x1875453360383488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5124us (5417us total) trans 4294970437 rc 0/0 00000100:00100000:0.0:1788572543.128271:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4177 00000100:00100000:0.0:1788572543.130567:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.131480:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360383744 00000100:00100000:1.0:1788572543.131492:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360383744 00000100:00100000:1.0:1788572543.131553:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4178 00000100:00100000:1.0:1788572543.131563:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35917:x1875453360383744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.131622:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572543.131650:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162ae6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efb9e expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131676:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627e7451a00) 00010000:00010000:1.0:1788572543.131697:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162b10 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efba5 expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131705:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572543.131708:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.131732:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131744:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131753:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131767:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131773:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.131778:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132544:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132555:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132563:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132568:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.132573:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132580:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132591:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572543.132594:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.132599:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132605:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b3a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132615:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132621:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132626:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132631:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.132634:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132643:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132650:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x214985f3 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.132652:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.132655:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.132661:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb162b41 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x81:0x0].0x214985f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.132797:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35917:x1875453360383744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1233us (2237us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572543.132805:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4178 00000100:00100000:0.0:1788572543.134693:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.134792:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360384000 00000100:00100000:2.0:1788572543.134803:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360384000 00000100:00100000:2.0:1788572543.134861:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4179 00000100:00100000:2.0:1788572543.134872:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35918:x1875453360384000:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.134981:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.134998:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.135029:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572543.135826:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].63, 88 off13256 00000004:00010000:2.0:1788572543.135938:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9f629c5 reply state 00000000ed242d78 transno 4294970438 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.135948:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572543.136075:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35918:x1875453360384000:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1203us (1387us total) trans 4294970438 rc 0/0 00000100:00100000:2.0:1788572543.136084:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4179 00000100:00100000:3.0:1788572543.137716:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.140481:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.140789:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.140842:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360384128 00000100:00100000:2.0:1788572543.140852:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360384128 00000100:00100000:2.0:1788572543.140903:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360384256 00000100:00100000:2.0:1788572543.140906:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360384256 00000100:00100000:2.0:1788572543.140941:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4180 00000100:00100000:2.0:1788572543.140950:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35919:x1875453360384128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.140975:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.140996:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162b4f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbba expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.141084:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.141098:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.141109:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.141122:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.141128:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.141133:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141649:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141659:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141665:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141672:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.141676:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141683:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141697:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:3.0:1788572543.141700:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1788572543.141705:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141711:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162b48 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.141746:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4181 00000100:00100000:3.0:1788572543.141757:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35917:x1875453360384256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.141767:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.141780:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162b64 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbc1 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141802:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141809:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141817:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141911:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162b64 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141937:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.141945:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572543.141948:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572543.141950:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572543.142027:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35917:x1875453360384256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 272us (1555us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.142034:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4181 00010000:00010000:2.0:1788572543.143718:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162b72 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143737:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162b72 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143747:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162b72 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572543.143782:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:2.0:1788572543.143786:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572543.143806:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162b72 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143820:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143827:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143834:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143839:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.143843:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143850:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143863:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572543.143867:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.143872:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143879:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162b56 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143892:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143897:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143903:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143907:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.143909:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143914:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.143920:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572543.143922:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.144088:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.144097:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b5d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.144138:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162b4f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbba expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.144152:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162b72 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbba expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.144158:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572543.144160:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572543.144163:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05a00, rc 0) 00000100:00100000:2.0:1788572543.144240:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35919:x1875453360384128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3290us (6533us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.144249:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4180 00000100:00100000:2.0:1788572543.146089:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360384384 00000100:00100000:2.0:1788572543.146102:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360384384 00000100:00100000:2.0:1788572543.146152:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4182 00000100:00100000:2.0:1788572543.146161:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35918:x1875453360384384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.146184:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.146214:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbc8 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.146270:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.146283:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.146484:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:2.0:1788572543.146487:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572543.146492:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572543.146544:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35918:x1875453360384384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 383us (5754us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.146551:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4182 00000100:00100000:3.0:1788572543.152610:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.153261:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.153508:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.153588:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360385024 00000100:00100000:1.0:1788572543.153599:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360385024 00000100:00100000:1.0:1788572543.153658:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1113 00000100:00100000:1.0:1788572543.153668:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7662850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360385024:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.153983:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7662850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360385024:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 316us (475us total) trans 4294967591 rc 0/0 00000100:00100000:1.0:1788572543.153991:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1113 00000100:00100000:0.0:1788572543.156687:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360384640 00000100:00100000:0.0:1788572543.156705:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360384640 00000100:00100000:0.0:1788572543.156785:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360384896 00000100:00100000:0.0:1788572543.156789:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360384896 00000100:00100000:0.0:1788572543.156819:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4183 00000100:00100000:0.0:1788572543.156829:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35917:x1875453360384640:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.156977:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4184 00000100:00100000:1.0:1788572543.156989:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35919:x1875453360384896:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.157059:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572543.157089:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162b2c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbb3 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157116:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7451100) 00010000:00010000:1.0:1788572543.157135:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162b72 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbba expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157144:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.157148:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.157171:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157185:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157195:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157210:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157217:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157223:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157512:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157519:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157529:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157535:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.157539:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157546:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157554:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.157557:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.157561:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157568:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162b80 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157578:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157584:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157589:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157594:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.157596:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157606:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157612:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xdfcb1c34 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.157614:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.157617:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.157623:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162b87 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0xdfcb1c34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.157708:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35919:x1875453360384896:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 722us (4448us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572543.157715:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4184 00010000:00010000:0.0:1788572543.158075:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.158087:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.158096:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.160584:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.160667:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360385280 00000100:00100000:1.0:1788572543.160675:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360385280 00000100:00100000:1.0:1788572543.160737:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 902 00000100:00100000:1.0:1788572543.160747:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35918:x1875453360385280:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.161140:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35918:x1875453360385280:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 392us (563us total) trans 4294970439 rc 0/0 00000100:00100000:1.0:1788572543.161149:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 902 00000040:00080000:0.0:1788572543.161377:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].61, 88 off13152 00000004:00010000:0.0:1788572543.161472:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000498def7b reply state 0000000078b7ef3c transno 4294970440 ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.161483:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572543.161544:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35917:x1875453360384640:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 4716us (8945us total) trans 4294970440 rc 0/0 00000100:00100000:0.0:1788572543.161550:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4183 00000100:00100000:1.0:1788572543.163697:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.164525:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360385408 00000100:00100000:2.0:1788572543.164537:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360385408 00000100:00100000:2.0:1788572543.164602:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4185 00000100:00100000:2.0:1788572543.164614:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35919:x1875453360385408:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.164636:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.164657:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162b95 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbcf expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.164701:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.164716:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.164728:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.164838:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162b95 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.164859:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 94 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.164868:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572543.164871:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572543.164875:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572543.166670:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35919:x1875453360385408:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2049us (2972us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.166683:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4185 00000100:00100000:3.0:1788572543.167498:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.167590:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.167619:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360385664 00000100:00100000:3.0:1788572543.167627:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360385664 00000100:00100000:3.0:1788572543.167672:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360385792 00000100:00100000:3.0:1788572543.167675:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360385792 00000100:00100000:3.0:1788572543.167698:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4186 00000100:00100000:3.0:1788572543.167707:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35918:x1875453360385664:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572543.167806:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35918:x1875453360385664:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 99us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.167818:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4186 00000100:00100000:3.0:1788572543.167835:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4187 00000100:00100000:3.0:1788572543.167842:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35917:x1875453360385792:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.167857:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.167881:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbd6 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.167952:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.167970:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.167990:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572543.167994:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572543.167997:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd200, rc 0) 00000100:00100000:3.0:1788572543.168046:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35917:x1875453360385792:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 205us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.168052:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4187 00010000:00010000:2.0:1788572543.168607:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.168617:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.168625:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.168630:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.168635:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.168643:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.168657:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:2.0:1788572543.168660:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572543.168664:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.168670:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162b8e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.173098:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.173182:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360385920 00000100:00100000:2.0:1788572543.173192:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360385920 00000100:00100000:2.0:1788572543.173256:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4188 00000100:00100000:2.0:1788572543.173265:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35918:x1875453360385920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.173286:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.173316:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162baa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbdd expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.173391:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162bb1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.173405:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.173416:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572543.173489:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572543.173514:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.173542:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162baa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbdd expref: 96 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.173560:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162bb1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbdd expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.173568:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:2.0:1788572543.173571:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572543.173575:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04600, rc 0) 00000100:00100000:2.0:1788572543.173639:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35918:x1875453360385920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 374us (551us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.173646:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4188 00000100:00100000:1.0:1788572543.178542:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.178589:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.178760:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360386048 00000100:00100000:2.0:1788572543.178771:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360386048 00000100:00100000:2.0:1788572543.178831:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1114 00000100:00100000:2.0:1788572543.178842:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360386048:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572543.179180:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360386048:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 340us (649us total) trans 4294967592 rc 0/0 00000100:00100000:2.0:1788572543.179188:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1114 00000100:00100000:1.0:1788572543.180907:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.180990:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360386432 00000100:00100000:0.0:1788572543.180997:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360386432 00000100:00100000:0.0:1788572543.181079:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1115 00000100:00100000:0.0:1788572543.181087:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360386432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.181102:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.181120:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbe4 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572543.181144:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737e, blocks: 0x1 00010000:00010000:0.0:1788572543.181161:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=891s v=(0 0) 00010000:00010000:0.0:1788572543.181165:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.181174:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 12 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.181190:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 12 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.181213:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:0.0:1788572543.181256:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360386432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 169us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.181262:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1115 00000100:00100000:3.0:1788572543.182182:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360386304 00000100:00100000:3.0:1788572543.182191:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360386304 00000100:00100000:3.0:1788572543.182384:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4189 00000100:00100000:3.0:1788572543.182394:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35919:x1875453360386304:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.182500:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.182519:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.182529:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572543.182876:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].64, 88 off13344 00000004:00010000:3.0:1788572543.182969:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000765d74b5 reply state 00000000b1bd6acc transno 4294970441 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.182979:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572543.183089:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35919:x1875453360386304:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 696us (4498us total) trans 4294970441 rc 0/0 00000100:00100000:3.0:1788572543.183097:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4189 00000100:00100000:1.0:1788572543.187378:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572543.187423:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.187435:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.187443:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.187449:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.187463:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.187472:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.187489:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572543.187492:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572543.187497:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 2/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.187503:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162bbf lrc: 0/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.189101:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.189376:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360386560 00000100:00100000:0.0:1788572543.189387:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360386560 00000100:00100000:0.0:1788572543.189447:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4190 00000100:00100000:0.0:1788572543.189471:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35919:x1875453360386560:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.189496:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.189517:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbeb expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.189583:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 97 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.189599:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 97 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.189621:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572543.189624:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572543.189627:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:0.0:1788572543.189890:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35919:x1875453360386560:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 421us (2521us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.189899:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4190 00000100:00100000:1.0:1788572543.192796:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.192871:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360387072 00000100:00100000:0.0:1788572543.192881:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360387072 00000100:00100000:0.0:1788572543.192946:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 903 00000100:00100000:0.0:1788572543.192958:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35917:x1875453360387072:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.193619:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35917:x1875453360387072:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 661us (830us total) trans 4294970442 rc 0/0 00000100:00100000:0.0:1788572543.193629:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 903 00000100:00100000:3.0:1788572543.193777:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360386816 00000100:00100000:3.0:1788572543.193783:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360386816 00000100:00100000:3.0:1788572543.193826:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4191 00000100:00100000:3.0:1788572543.193833:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35918:x1875453360386816:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572543.193888:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35918:x1875453360386816:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 56us (4796us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.193895:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4191 00000100:00100000:3.0:1788572543.198280:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.198526:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.198575:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.198604:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360387584 00000100:00100000:3.0:1788572543.198613:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360387584 00000100:00100000:1.0:1788572543.198721:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360387200 00000100:00100000:1.0:1788572543.198726:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360387200 00000100:00100000:3.0:1788572543.198787:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1116 00000100:00100000:3.0:1788572543.198797:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35918:x1875453360387584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.198819:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.198838:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbf2 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.198885:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.198894:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572543.198898:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572543.198903:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.198950:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.198963:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.198976:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5433b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382358528:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.199073:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.199234:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360387456 00000100:00100000:0.0:1788572543.199239:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360387456 00000100:00100000:0.0:1788572543.199278:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4192 00000100:00100000:0.0:1788572543.199285:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35917:x1875453360387456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.199366:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35917:x1875453360387456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 81us (1096us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.199374:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4192 00000100:00100000:1.0:1788572543.205607:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1117 00000100:00100000:1.0:1788572543.205626:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360387200:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.206446:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360387200:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 823us (7865us total) trans 4294967592 rc 0/0 00000100:00100000:1.0:1788572543.206466:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1117 00000100:00100000:0.0:1788572543.209613:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5433b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382358528:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.209667:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.209690:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:0.0:1788572543.211059:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35918:x1875453360387584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 12253us (12523us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.211074:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1116 00000100:00100000:2.0:1788572543.214864:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.214902:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.214982:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.215165:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360388096 00000100:00100000:3.0:1788572543.215177:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360388096 00000100:00100000:3.0:1788572543.215253:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 904 00000100:00100000:3.0:1788572543.215264:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35919:x1875453360388096:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572543.215852:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35919:x1875453360388096:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 589us (869us total) trans 4294970443 rc 0/0 00000100:00100000:3.0:1788572543.215861:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 904 00000100:00100000:1.0:1788572543.216112:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360387712 00000100:00100000:1.0:1788572543.216118:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360387712 00010000:00010000:1.0:1788572543.216145:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.216179:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 992 00000100:00100000:1.0:1788572543.216185:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360387712:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572543.216197:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.216218:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.216229:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.216251:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162bb8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbe4 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.216272:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.216277:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.216477:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.216490:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.216532:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.216547:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7874a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382358656:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572543.216606:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d56f8600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.216744:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360387840 00000100:00100000:0.0:1788572543.216749:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360387840 00000100:00100000:0.0:1788572543.216835:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4193 00000100:00100000:0.0:1788572543.216843:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35917:x1875453360387840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.216855:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.216874:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162bd4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efbf9 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.216951:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162bdb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.216965:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162bdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.216979:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162bdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.217030:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.217048:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162bdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.217070:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162bd4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf9 expref: 101 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.217084:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162bdb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf9 expref: 100 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.217091:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572543.217094:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572543.217097:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47e00, rc 0) 00000100:00100000:0.0:1788572543.217123:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35917:x1875453360387840:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 282us (2221us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.217129:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4193 00000100:00100000:1.0:1788572543.222616:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7874a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382358656:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572543.222669:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572543.222746:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360387712:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6558us (7890us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.222759:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 992 00000100:00100000:3.0:1788572543.227498:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.229896:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360388352 00000100:00100000:0.0:1788572543.229913:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360388352 00000100:00100000:0.0:1788572543.229999:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4194 00000100:00100000:0.0:1788572543.230032:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35919:x1875453360388352:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.230161:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35919:x1875453360388352:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 131us (2672us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.230168:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4194 00000100:00100000:2.0:1788572543.233708:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.233757:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.233895:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360388608 00000100:00100000:1.0:1788572543.233906:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360388608 00000100:00100000:1.0:1788572543.233968:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1118 00000100:00100000:1.0:1788572543.233979:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360388608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.233998:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.234048:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc00 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572543.234078:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737e, blocks: 0x1 00010000:00010000:1.0:1788572543.234098:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=891s v=(0 0) 00010000:00010000:1.0:1788572543.234102:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.234111:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.234127:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.234139:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678400, rc 0) 00000100:00100000:1.0:1788572543.234187:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360388608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 211us (430us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.234194:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1118 00000100:00100000:0.0:1788572543.234266:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360388480 00000100:00100000:0.0:1788572543.234271:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360388480 00002000:00100000:0.0:1788572543.234313:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4b:0x0] (0->2) req@ffff9626c36ded80 x1875453360388480/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/0 e 0 to 0 dl 1788572554 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572543.234322:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36ded80 with x1875453360388480 ext(0->2) 00002000:00010000:0.0:1788572543.234327:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36ded80 00000100:00100000:0.0:1788572543.234351:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 254 00000100:00100000:0.0:1788572543.234359:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360388480:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.234793:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360388480:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 434us (1095us total) trans 4294967593 rc 0/0 00000100:00100000:0.0:1788572543.234800:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 254 00002000:00100000:0.0:1788572543.234806:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4b:0x0] (0->2) req@ffff9626c36ded80 x1875453360388480/t4294967593(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/448 e 0 to 0 dl 1788572554 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572543.234813:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36ded80 with x1875453360388480 ext(0->2) 00002000:00010000:0.0:1788572543.234816:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36ded80 00000100:00100000:1.0:1788572543.238302:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.238467:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360388736 00000100:00100000:2.0:1788572543.238480:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360388736 00000100:00100000:2.0:1788572543.238545:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4195 00000100:00100000:2.0:1788572543.238557:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35919:x1875453360388736:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.238582:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.238605:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162be9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc07 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.238791:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162bf0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.238808:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.238821:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572543.238865:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572543.238889:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.238921:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162be9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc07 expref: 102 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.238949:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162bf0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc07 expref: 101 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.238961:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572543.238964:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572543.238968:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05400, rc 0) 00000100:00100000:2.0:1788572543.239164:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35919:x1875453360388736:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 609us (878us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.239172:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4195 00000100:00100000:3.0:1788572543.243977:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.244161:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.248436:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.248565:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360389248 00000100:00100000:1.0:1788572543.248575:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360389248 00000100:00100000:1.0:1788572543.248636:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1119 00000100:00100000:1.0:1788572543.248659:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360389248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.248678:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.248701:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc0e expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572543.248729:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737e, blocks: 0x1 00010000:00010000:1.0:1788572543.248749:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=891s v=(0 0) 00010000:00010000:1.0:1788572543.248754:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.248762:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 16 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.248781:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 16 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.248793:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679800, rc 0) 00000100:00100000:1.0:1788572543.248853:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360389248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 196us (425us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.248861:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1119 00000100:00100000:1.0:1788572543.252047:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.252430:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360388864 00000100:00100000:2.0:1788572543.252440:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360388864 00000100:00100000:0.0:1788572543.252568:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360389120 00000100:00100000:0.0:1788572543.252573:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360389120 00000100:00100000:0.0:1788572543.252634:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4196 00000100:00100000:0.0:1788572543.252654:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35917:x1875453360389120:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.252896:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35917:x1875453360389120:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 243us (8733us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.252905:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4196 00000100:00100000:0.0:1788572543.253522:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360389504 00000100:00100000:0.0:1788572543.253528:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360389504 00000100:00100000:0.0:1788572543.253553:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4197 00000100:00100000:0.0:1788572543.253560:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35918:x1875453360388864:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.254804:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4198 00000100:00100000:0.0:1788572543.254815:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35919:x1875453360389504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.255373:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35919:x1875453360389504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 111us (2923us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.255383:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4198 00000100:00100000:1.0:1788572543.270021:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.270056:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360389632 00000100:00100000:1.0:1788572543.270067:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360389632 00000100:00100000:1.0:1788572543.270118:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1120 00000100:00100000:1.0:1788572543.270128:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35917:x1875453360389632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.270146:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.270164:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc15 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.270195:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.270216:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.270220:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.270227:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.270264:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.270280:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.270443:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382358784:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572543.270546:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d56f8600 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.274646:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.274791:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360389760 00000100:00100000:0.0:1788572543.274802:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360389760 00000100:00100000:0.0:1788572543.274861:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1121 00000100:00100000:0.0:1788572543.274872:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35919:x1875453360389760:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.274893:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.274911:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc1c expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.274956:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.274965:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.274970:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.274975:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 17 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.275045:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.275057:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.275083:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382358912:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.275157:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9932d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.286951:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382358912:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.286999:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 17 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.287048:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:0.0:1788572543.287107:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35919:x1875453360389760:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 12236us (12480us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.287116:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1121 00000100:00100000:1.0:1788572543.288248:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382358784:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.288275:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.288299:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679400, rc 0) 00000100:00100000:1.0:1788572543.288338:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35917:x1875453360389632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 18211us (18345us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.288346:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1120 00000100:00100000:0.0:1788572543.289359:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.289693:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.289721:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390016 00000100:00100000:0.0:1788572543.289728:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390016 00010000:00010000:0.0:1788572543.289764:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.289795:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360389888 00000100:00100000:0.0:1788572543.289797:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360389888 00010000:00010000:0.0:1788572543.289810:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.289832:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 993 00000100:00100000:0.0:1788572543.289841:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360390016:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.289875:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.289899:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.289911:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.289944:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f78b3a39/0x85ace67feb162bf7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc0e expref: 17 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.289962:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.289967:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.289976:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.289986:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.290074:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.290090:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2734380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382359040:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572543.290153:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9fad80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.290199:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382359168:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788572543.290234:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.290272:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382359168 00000100:00100000:0.0:1788572543.290275:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382359168 00000100:00100000:0.0:1788572543.290316:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1788572543.290324:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382359168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788572543.290356:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 166658048 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788572543.290397:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382359168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.290404:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1788572543.290464:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382359168:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788572543.290493:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 994 00000100:00100000:0.0:1788572543.290500:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e51850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360389888:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.290508:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.290521:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.290533:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.290547:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a1c3ce44/0x85ace67feb162be2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc00 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.290561:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.290564:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.290570:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.290583:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.290607:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.290617:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382359296:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572543.290642:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9932d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.300220:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.300359:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2734380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382359040:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572543.300381:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.300434:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360390016:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 10593us (11078us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.300447:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 993 00000100:00100000:1.0:1788572543.302094:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390144 00000100:00100000:1.0:1788572543.302111:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390144 00002000:00100000:1.0:1788572543.302167:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4b:0x0] (0->2) req@ffff9627e7b9fb80 x1875453360390144/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/0 e 0 to 0 dl 1788572554 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.302179:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7b9fb80 with x1875453360390144 ext(0->2) 00002000:00010000:1.0:1788572543.302186:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7b9fb80 00000100:00100000:1.0:1788572543.302223:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 255 00000100:00100000:1.0:1788572543.302233:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360390144:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.305045:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360390144:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 1053us (3084us total) trans 4294967593 rc 0/0 00000100:00100000:1.0:1788572543.305065:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 255 00002000:00100000:1.0:1788572543.305080:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4b:0x0] (0->2) req@ffff9627e7b9fb80 x1875453360390144/t4294967593(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/448 e 0 to 0 dl 1788572554 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.305090:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7b9fb80 with x1875453360390144 ext(0->2) 00002000:00010000:1.0:1788572543.305095:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7b9fb80 00000100:00100000:0.0:1788572543.305273:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.305824:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390272 00000100:00100000:1.0:1788572543.305834:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390272 00002000:00100000:1.0:1788572543.305877:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4c:0x0] (0->2) req@ffff9627e7b9ca80 x1875453360390272/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/0 e 0 to 0 dl 1788572554 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.305888:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7b9ca80 with x1875453360390272 ext(0->2) 00002000:00010000:1.0:1788572543.305892:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e7b9ca80 00000100:00100000:1.0:1788572543.305930:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 256 00000100:00100000:1.0:1788572543.305940:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360390272:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572543.311499:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382359296:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572543.311535:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.311628:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e51850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360389888:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 21126us (21933us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.311867:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360390272:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 5927us (6593us total) trans 4294967594 rc 0/0 00000100:00100000:0.0:1788572543.311879:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 256 00002000:00100000:0.0:1788572543.311893:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4c:0x0] (0->2) req@ffff9627e7b9ca80 x1875453360390272/t4294967594(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/448 e 0 to 0 dl 1788572554 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572543.311904:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7b9ca80 with x1875453360390272 ext(0->2) 00002000:00010000:0.0:1788572543.311910:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e7b9ca80 00000100:00100000:2.0:1788572543.312105:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.312470:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 994 00000100:00100000:0.0:1788572543.312624:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390400 00000100:00100000:0.0:1788572543.312633:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390400 00000100:00100000:0.0:1788572543.313024:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4199 00000100:00100000:0.0:1788572543.313035:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35917:x1875453360390400:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.313141:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35917:x1875453360390400:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 108us (1036us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.313147:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4199 00000100:00100000:2.0:1788572543.320323:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.320463:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390528 00000100:00100000:1.0:1788572543.320474:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390528 00000100:00100000:1.0:1788572543.320537:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4200 00000100:00100000:1.0:1788572543.320548:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35919:x1875453360390528:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.320687:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35919:x1875453360390528:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 140us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.320698:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4200 00000100:00100000:1.0:1788572543.324666:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.324716:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.324856:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390656 00000100:00100000:2.0:1788572543.324867:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390656 00000100:00100000:2.0:1788572543.324935:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390784 00000100:00100000:2.0:1788572543.324939:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390784 00000100:00100000:2.0:1788572543.324962:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1122 00000100:00100000:2.0:1788572543.324972:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360390656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.325161:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1123 00000100:00100000:1.0:1788572543.325173:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360390784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1788572543.345054:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970442 is committed 00000020:00080000:2.0:1788572543.345086:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ca85e840 refcount 2 committed 0 result 0 batchid 114 00000020:00080000:2.0:1788572543.345092:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3100 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff5483c0 00000020:00080000:2.0:1788572543.345095:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffd54c00 00000020:00080000:2.0:1788572543.345109:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c33f7000 refcount 2 committed 0 result 0 batchid 113 00000020:00080000:2.0:1788572543.345112:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3900 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c33f7240 00000020:00080000:2.0:1788572543.345116:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffe3f000 00000020:00080000:2.0:1788572543.345132:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 111 batchid updated 00000020:00080000:2.0:1788572543.345136:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 109 batchid updated 00000020:00080000:2.0:1788572543.345158:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d1317c00 refcount 2 committed 0 result 0 batchid 112 00000020:00080000:2.0:1788572543.345161:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8adc00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d13176c0 00000020:00080000:2.0:1788572543.345164:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c10abc00 00000001:00080000:2.0:1788572543.345189:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970443 is committed 00000100:00100000:0.0:1788572543.345515:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35918:x1875453360388864:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 91947us (101540us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.345527:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4197 00000020:00080000:0.0:1788572543.345562:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 114 committed batchid 111 00000020:00080000:0.0:1788572543.347505:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 111 00000100:00100000:1.0:1788572543.353905:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.354468:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360390912 00000100:00100000:3.0:1788572543.354480:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360390912 00000100:00100000:3.0:1788572543.354540:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1124 00000100:00100000:3.0:1788572543.354549:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360390912:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572543.354688:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360390912:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 139us (795us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.354696:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1124 00000100:00100000:3.0:1788572543.364439:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.364489:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360391168 00000100:00100000:3.0:1788572543.364499:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360391168 00000100:00100000:3.0:1788572543.364561:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4201 00000100:00100000:3.0:1788572543.364572:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35918:x1875453360391168:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.364760:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.364790:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb162bb1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbdd expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.364817:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:3.0:1788572543.364862:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572543.364900:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.364916:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.364927:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572543.365262:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b347d5ca reply state 000000001bb052cf transno 4294970444 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.365274:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572543.365392:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35918:x1875453360391168:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 821us (959us total) trans 4294970444 rc 0/0 00000100:00100000:3.0:1788572543.365400:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4201 00010000:00010000:1.0:1788572543.371684:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.371703:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.371711:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.371717:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.371722:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.371730:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.371746:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:1.0:1788572543.371749:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.371755:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.371761:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162c0c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.374998:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.375347:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360391296 00000100:00100000:0.0:1788572543.375358:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360391296 00000100:00100000:0.0:1788572543.375414:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1125 00000100:00100000:0.0:1788572543.375424:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360391296:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572543.375669:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4b:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572543.375673:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4b:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:0.0:1788572543.375676:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4b:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:0.0:1788572543.375678:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4b:0x0] updating lvb ctime from disk: 1788572542 -> 1788572527 00002000:00010000:0.0:1788572543.375680:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4b:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572543.375736:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+20:2349:x1875453360391296:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 314us (747us total) trans 4294967595 rc 0/0 00000100:00100000:0.0:1788572543.375743:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1125 00000100:00100000:1.0:1788572543.379542:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.379778:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360391552 00000100:00100000:0.0:1788572543.379790:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360391552 00000100:00100000:0.0:1788572543.379852:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4202 00000100:00100000:0.0:1788572543.379864:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35918:x1875453360391552:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.379947:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.379965:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.379975:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.380043:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.380062:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 65 pid: 5838 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.380092:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382359424:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.380180:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.383862:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382359424:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.383893:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.387992:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.388148:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360391680 00000100:00100000:3.0:1788572543.388161:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360391680 00010000:00010000:3.0:1788572543.388201:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 66 pid: 5838 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.388239:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 995 00000100:00100000:3.0:1788572543.388249:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360391680:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.388263:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.388271:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 66 pid: 5838 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.388285:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 66 pid: 5838 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.388519:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162b1e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x87:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbac expref: 66 pid: 5838 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.388536:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:3.0:1788572543.388541:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.388550:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.388557:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.388589:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572543.388652:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388661:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388692:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388700:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388707:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388713:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.388718:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388725:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388739:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:2.0:1788572543.388742:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572543.388745:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.388750:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162c13 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.388804:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35918:x1875453360391552:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8941us (9273us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572543.388813:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4202 00000100:00100000:3.0:1788572543.389109:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360391680:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 861us (1127us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.389117:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 995 00000100:00100000:2.0:1788572543.389342:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360390656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 64370us (64685us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.389351:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1122 00000001:00080000:0.0:1788572543.389493:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967593 is committed 00000001:00080000:0.0:1788572543.389512:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967594 is committed 00000001:00080000:0.0:1788572543.389524:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967590 is committed 00000100:00100000:1.0:1788572543.393937:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.394124:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360391936 00000100:00100000:3.0:1788572543.394135:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360391936 00000100:00100000:3.0:1788572543.394201:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4203 00000100:00100000:3.0:1788572543.394212:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35919:x1875453360391936:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.394273:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.394312:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162bf0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc07 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.394344:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:3.0:1788572543.394350:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572543.394383:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.394398:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.394408:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572543.394577:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004803514a reply state 0000000088042abe transno 4294970445 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.394586:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572543.394697:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35919:x1875453360391936:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 486us (780us total) trans 4294970445 rc 0/0 00000100:00100000:3.0:1788572543.394704:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4203 00000100:00100000:1.0:1788572543.398627:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.398663:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360392064 00000100:00100000:1.0:1788572543.398674:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360392064 00000100:00100000:1.0:1788572543.398727:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1126 00000100:00100000:1.0:1788572543.398738:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360392064:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.398783:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.398806:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 12 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.398815:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.398820:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572543.398824:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399409:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399420:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399426:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399435:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.399440:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399447:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399473:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572543.399476:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.399481:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.399487:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162c1a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.399528:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.404356:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360392192 00000100:00100000:1.0:1788572543.404373:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360392192 00000100:00100000:1.0:1788572543.404479:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1127 00000100:00100000:1.0:1788572543.404491:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360392192:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572543.404690:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4c:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572543.404694:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4c:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:1.0:1788572543.404697:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4c:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:1.0:1788572543.404699:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4c:0x0] updating lvb ctime from disk: 1788572542 -> 1788572527 00002000:00010000:1.0:1788572543.404701:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4c:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572543.404775:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360392192:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 286us (5246us total) trans 4294967596 rc 0/0 00000100:00100000:1.0:1788572543.404783:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1127 00010000:00010000:1.0:1788572543.407570:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.407596:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 13 pid: 6679 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.407616:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382359552:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572543.407750:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f600 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.411668:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.411806:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360392448 00000100:00100000:3.0:1788572543.411820:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360392448 00000100:00100000:3.0:1788572543.411877:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4204 00000100:00100000:3.0:1788572543.411887:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35919:x1875453360392448:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.411962:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.411979:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.411994:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.412472:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.412486:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 64 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.412500:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382359680:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.412568:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1317840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.420583:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360390784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 95400us (95853us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.420598:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1123 00000001:00080000:0.0:1788572543.420647:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967591 is committed 00000001:00080000:0.0:1788572543.420655:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967590 is committed 00000001:00080000:0.0:1788572543.420664:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967593 is committed 00000100:00100000:1.0:1788572543.421127:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.421188:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382359680:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.421223:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.421267:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.421759:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360392576 00000100:00100000:0.0:1788572543.421767:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360392576 00010000:00010000:0.0:1788572543.421803:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 14 pid: 6679 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.421834:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360392704 00000100:00100000:0.0:1788572543.421838:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360392704 00010000:00010000:0.0:1788572543.421866:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 65 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.421891:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 996 00000100:00100000:0.0:1788572543.421901:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360392576:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.421916:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.421954:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 14 pid: 6679 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.421970:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 14 pid: 6679 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.421998:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572543.422021:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572543.422028:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422039:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422059:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.422114:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360392576:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 214us (988us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.422122:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 996 00000100:00100000:0.0:1788572543.422143:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 997 00000100:00100000:0.0:1788572543.422150:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360392704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.422162:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.422169:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 65 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422180:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 65 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422196:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162b9c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x89:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbcf expref: 65 pid: 22536 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422225:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572543.422231:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422241:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422249:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422269:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.422290:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360392704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 140us (1022us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.422297:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 997 00010000:00010000:0.0:1788572543.422326:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422334:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422373:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422382:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422390:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422397:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.422403:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422411:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422421:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:0.0:1788572543.422424:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572543.422429:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.422435:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162c28 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.422490:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35919:x1875453360392448:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10604us (10831us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572543.422499:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4204 00010000:00010000:3.0:1788572543.424750:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb162bcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf2 expref: 13 pid: 6679 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.424776:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382359552:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572543.424800:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.424813:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.424821:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425106:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425117:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425125:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425132:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572543.425136:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425145:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425165:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.425173:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fdf7934b/0x85ace67feb162c21 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.425331:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360392064:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 26593us (26713us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.425342:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1126 00000100:00100000:2.0:1788572543.431746:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.431799:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.431882:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360392832 00000100:00100000:3.0:1788572543.431900:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360392832 00000100:00100000:3.0:1788572543.431990:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1128 00000100:00100000:3.0:1788572543.432023:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360392832:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.432076:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.432099:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.432108:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572543.432113:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.432118:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 4/1,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.432154:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.432169:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 12 pid: 13272 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.432185:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382359808:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.432293:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1317840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.432981:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360393088 00000100:00100000:1.0:1788572543.432989:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360393088 00000100:00100000:1.0:1788572543.433058:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4205 00000100:00100000:1.0:1788572543.433068:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35918:x1875453360393088:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.433133:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.433147:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.433159:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.436631:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.436661:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 64 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.436672:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 4/0,1 mode: --/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.436713:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 64 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.436727:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 64 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.436742:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382359936:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572543.436837:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.437890:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.438169:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360393344 00000100:00100000:1.0:1788572543.438178:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360393344 00000100:00100000:1.0:1788572543.438243:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4206 00000100:00100000:1.0:1788572543.438254:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35917:x1875453360393344:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.438303:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.438329:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162bdb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbf9 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.438346:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:1.0:1788572543.438351:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.438378:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.438392:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.438402:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572543.438550:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7544ddd reply state 000000003426a59f transno 4294970447 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.438559:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572543.438651:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35917:x1875453360393344:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 398us (765us total) trans 4294970447 rc 0/0 00000100:00100000:1.0:1788572543.438659:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4206 00000100:00100000:2.0:1788572543.440663:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382359808:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572543.440693:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443095:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443112:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443123:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443130:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.443134:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443143:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443159:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572543.443162:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.443167:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.443173:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162c44 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.444373:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.444503:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360393472 00000100:00100000:3.0:1788572543.444514:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360393472 00010000:00010000:3.0:1788572543.444558:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 13 pid: 13272 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.444603:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 998 00000100:00100000:3.0:1788572543.444614:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360393472:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.444630:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.444656:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 13 pid: 13272 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.444675:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 13 pid: 13272 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.444697:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3ee15d6/0x85ace67feb162c05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc1c expref: 13 pid: 13272 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.444713:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.444717:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.444724:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.444734:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.444773:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.444831:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360393472:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 219us (464us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.444838:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 998 00010000:00010000:2.0:1788572543.444911:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.444932:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.444962:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.444970:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.444977:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.444984:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572543.444988:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.444997:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.445042:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.445051:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb162c2f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.445098:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360392832:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 13097us (13366us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.445109:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1128 00000100:00100000:1.0:1788572543.449088:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.449218:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360393600 00000100:00100000:0.0:1788572543.449229:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360393600 00000100:00100000:0.0:1788572543.449294:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1129 00000100:00100000:0.0:1788572543.449304:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360393600:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572543.449571:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4b:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572543.449575:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4b:0x0] updating lvb mtime from disk: 0 -> 1788572520 00002000:00010000:0.0:1788572543.449578:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4b:0x0] updating lvb atime from disk: 0 -> 1788572520 00002000:00010000:0.0:1788572543.449580:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4b:0x0] updating lvb ctime from disk: 1788572542 -> 1788572524 00002000:00010000:0.0:1788572543.449582:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4b:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572543.449642:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360393600:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 339us (564us total) trans 4294967594 rc 0/0 00000100:00100000:0.0:1788572543.449650:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1129 00000100:00100000:1.0:1788572543.453929:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.454765:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.457302:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.457944:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382359936:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.457979:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.458046:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360394240 00000100:00100000:1.0:1788572543.458053:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360394240 00010000:00010000:1.0:1788572543.458096:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 65 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.458141:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 999 00000100:00100000:1.0:1788572543.458151:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360394240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572543.458168:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.458174:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 65 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458187:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 65 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458222:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb162b79 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc8 expref: 65 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458241:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:1.0:1788572543.458246:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458257:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458265:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458288:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572543.458346:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360394240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 196us (1054us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.458354:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 999 00010000:00010000:1.0:1788572543.458376:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.458384:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572543.458593:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970446 00000001:00080000:1.0:1788572543.458755:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970448 00000001:00080000:1.0:1788572543.458802:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970449 00000100:00100000:1.0:1788572543.458892:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382360064:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572543.458965:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572543.459045:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.459070:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382360064 00000100:00100000:1.0:1788572543.459074:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382360064 00000100:00100000:1.0:1788572543.459110:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:1.0:1788572543.459119:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382360064:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572543.459187:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382360064:12345-0@lo:1000:mdt00_003.0 Request processed in 69us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.459193:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:1.0:1788572543.459228:0:9040:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967590 gen 2 00000004:00080000:1.0:1788572543.459234:0:9040:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967590 committed transno 4294967590 00000100:00100000:1.0:1788572543.459251:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382360064:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1788572543.459621:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970450 00000100:00100000:1.0:1788572543.459674:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382360192:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572543.459703:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572543.459724:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.459739:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382360192 00000100:00100000:1.0:1788572543.459743:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382360192 00000100:00100000:1.0:1788572543.459768:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1788572543.459776:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382360192:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572543.459933:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382360192:12345-0@lo:1000:mdt00_003.0 Request processed in 157us (229us total) trans 4294967597 rc 0/0 00000100:00100000:1.0:1788572543.459939:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1788572543.459972:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382360192:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1788572543.460000:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460032:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460041:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460047:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.460053:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460062:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460075:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7451000) 00010000:00010000:1.0:1788572543.460079:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572543.460084:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460092:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb162c3d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x87:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460127:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460134:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460140:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460145:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.460148:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460154:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460161:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:1.0:1788572543.460163:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572543.460166:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460172:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935600/0x85ace67feb15f483 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460194:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460206:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460214:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460220:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460226:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460232:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460237:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.460239:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460245:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460252:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:1.0:1788572543.460254:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.460256:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460262:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162c4b lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460271:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460276:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460281:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460286:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.460289:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460295:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460303:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:1.0:1788572543.460305:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572543.460308:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.460313:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb162c36 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.463771:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360393856 00000100:00100000:3.0:1788572543.463793:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360393856 00000100:00100000:0.0:1788572543.464154:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360394112 00000100:00100000:0.0:1788572543.464161:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360394112 00000100:00100000:3.0:1788572543.464308:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4207 00000100:00100000:3.0:1788572543.464321:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35919:x1875453360393856:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.464414:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464436:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464449:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464698:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464710:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464718:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464755:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.464769:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 70 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.464783:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382360320:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.465040:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4208 00000100:00100000:0.0:1788572543.465051:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35917:x1875453360394112:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.465086:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.465096:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 70 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.465103:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.465122:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 70 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.465131:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 70 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.465140:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382360448:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.465243:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1317840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.465603:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b0b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.465692:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35918:x1875453360393088:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 27381us (28649us total) trans 4294970451 rc 0/0 00000100:00100000:1.0:1788572543.465706:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4205 00000020:00080000:1.0:1788572543.465714:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:2.0:1788572543.465805:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.468761:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360394496 00000100:00100000:3.0:1788572543.468778:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360394496 00000100:00100000:3.0:1788572543.468914:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 905 00000100:00100000:3.0:1788572543.468986:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35918:x1875453360394496:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572543.470612:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x87:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572543.470733:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970453 00010000:00010000:2.0:1788572543.470796:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.470814:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.470828:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.470837:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.470844:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.471340:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.471405:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572543.471534:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35918:x1875453360394496:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 2606us (5725us total) trans 4294970454 rc 0/0 00000100:00100000:2.0:1788572543.471545:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 905 00000020:00080000:2.0:1788572543.471552:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788572543.471757:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.471785:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.472843:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382360320:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572543.472881:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.472911:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360394624 00000100:00100000:3.0:1788572543.472918:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360394624 00010000:00010000:3.0:1788572543.472962:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 72 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.472998:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360394752 00000100:00100000:3.0:1788572543.473024:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360394752 00010000:00010000:3.0:1788572543.473039:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 73 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.473141:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1000 00000100:00100000:3.0:1788572543.473152:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2350:x1875453360394624:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.473167:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.473175:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 73 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473187:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 73 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473220:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162bc6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbeb expref: 73 pid: 9040 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473235:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:3.0:1788572543.473240:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473254:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473261:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473286:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.473329:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2350:x1875453360394624:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 180us (1572us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.473337:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1000 00000100:00100000:3.0:1788572543.473352:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1001 00000100:00100000:3.0:1788572543.473360:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e50850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2350:x1875453360394752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.473368:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.473372:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 71 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473382:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 71 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473394:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572543.473399:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473406:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473411:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.473426:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.473445:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e50850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2350:x1875453360394752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 86us (1659us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.473461:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1001 00010000:00010000:0.0:1788572543.474038:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572543.474046:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474054:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572543.474058:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474067:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474077:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x87:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:0.0:1788572543.474079:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572543.474083:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474089:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162c67 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x87:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474107:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.474109:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.474110:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.474142:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162b6b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x88:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbc1 expref: 70 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.474153:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382360448:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.474173:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474181:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474186:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474225:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474233:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474239:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474244:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.474248:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474254:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474262:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572543.474265:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572543.474268:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.474275:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162c60 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.474329:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35917:x1875453360394112:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 9280us (19566us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572543.474339:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4208 00010000:00010000:3.0:1788572543.475480:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.475491:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572543.475682:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970452 00000001:00080000:3.0:1788572543.476023:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970455 00000001:00080000:3.0:1788572543.476078:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970456 00000100:00100000:3.0:1788572543.476175:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382360576:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572543.476316:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572543.476461:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda300 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.477722:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382360576 00000100:00100000:2.0:1788572543.477733:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382360576 00000100:00100000:2.0:1788572543.477790:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1788572543.477800:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382360576:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1788572543.477968:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382360576:12345-0@lo:1000:mdt00_005.0 Request processed in 169us (1651us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.477976:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1788572543.480591:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.480664:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360394880 00000100:00100000:0.0:1788572543.480673:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360394880 00000100:00100000:0.0:1788572543.480733:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1130 00000100:00100000:0.0:1788572543.480743:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360394880:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.480790:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.480813:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.480821:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.480826:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572543.480831:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 4/1,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.480869:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.480884:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 11 pid: 6682 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.480899:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382360704:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.480994:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b0b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.481783:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382360576:0@lo:1000:mdt00_005.0 00000001:00080000:0.0:1788572543.482532:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970457 00000100:00100000:0.0:1788572543.482618:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9c700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382360832:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572543.482674:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.482761:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f643c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.488225:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382360832 00000100:00100000:1.0:1788572543.488240:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382360832 00000100:00100000:0.0:1788572543.488430:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1788572543.488442:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382360832:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572543.488783:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:23058:x1875453382360832:12345-0@lo:1000:mdt00_005.0 Request processed in 342us (6108us total) trans 4294967598 rc 0/0 00000100:00100000:0.0:1788572543.488792:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:3.0:1788572543.488902:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9c700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382360832:0@lo:1000:mdt00_005.0 00010000:00010000:3.0:1788572543.488958:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.488968:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.488976:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.488982:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.488987:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.488995:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489038:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:3.0:1788572543.489042:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572543.489047:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489053:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c59 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x89:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489083:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489090:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489096:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489102:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.489105:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489111:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489120:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:3.0:1788572543.489122:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572543.489127:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489132:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb15f4d0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489152:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489165:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489174:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489181:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489186:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489191:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489196:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.489198:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489215:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489222:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:3.0:1788572543.489225:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.489228:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489234:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162c75 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489244:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489249:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489254:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489259:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.489261:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489267:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489275:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:3.0:1788572543.489277:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572543.489280:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.489286:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c52 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.489428:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35919:x1875453360393856:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 25108us (35514us total) trans 4294970458 rc 0/0 00000100:00100000:3.0:1788572543.489438:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4207 00000020:00080000:3.0:1788572543.489442:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:2.0:1788572543.493817:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.493915:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.493978:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360395264 00000100:00100000:1.0:1788572543.493986:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360395264 00010000:00010000:1.0:1788572543.494052:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 12 pid: 6682 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.494089:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1002 00000100:00100000:1.0:1788572543.494097:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360395264:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572543.494113:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.494133:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 12 pid: 6682 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.494143:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 12 pid: 6682 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.494163:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572543.494166:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572543.494171:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.494178:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.494192:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572543.494241:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360395264:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 145us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.494246:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1002 00000100:00100000:3.0:1788572543.494559:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360395136 00000100:00100000:3.0:1788572543.494564:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360395136 00000100:00100000:3.0:1788572543.494602:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 906 00000100:00100000:3.0:1788572543.494610:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd0c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35919:x1875453360395136:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572543.494969:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x89:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572543.495000:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970459 00010000:00010000:3.0:1788572543.495063:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.495077:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.495085:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.495090:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.495095:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.495099:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.495108:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572543.495159:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd0c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35919:x1875453360395136:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 550us (1354us total) trans 4294970460 rc 0/0 00000100:00100000:3.0:1788572543.495164:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 906 00000020:00080000:3.0:1788572543.495166:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572543.495475:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572543.495482:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.495489:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572543.495493:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.495501:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.495511:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x89:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:2.0:1788572543.495513:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572543.495516:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.495520:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162c7c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x89:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.495532:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572543.495533:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572543.495535:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.496704:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb162bfe lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc15 expref: 11 pid: 6682 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.496723:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382360704:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.496748:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496760:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496767:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496795:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496807:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496814:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496820:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572543.496824:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496831:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496843:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.496901:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b07f7d0/0x85ace67feb162c6e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.497192:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360394880:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 16448us (16608us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.497211:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1130 00000100:00100000:2.0:1788572543.499246:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.499758:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360395520 00000100:00100000:3.0:1788572543.499767:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360395520 00000100:00100000:3.0:1788572543.499815:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4209 00000100:00100000:3.0:1788572543.499825:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35917:x1875453360395520:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.499921:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.499938:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.499950:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.500305:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.500318:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 75 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.500327:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 4/0,1 mode: --/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.500362:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 75 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.500377:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 75 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.500391:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382360960:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572543.500488:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.503655:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.503863:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382360960:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572543.503892:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.504100:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360395648 00000100:00100000:1.0:1788572543.504108:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360395648 00010000:00010000:1.0:1788572543.504146:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 76 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.504255:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1003 00000100:00100000:1.0:1788572543.504265:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2349:x1875453360395648:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572543.504286:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.504293:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 76 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.504305:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 76 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.504326:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162ba3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efbd6 expref: 76 pid: 5839 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.504341:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:1.0:1788572543.504346:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.504355:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.504363:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.504393:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572543.504461:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360395648:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 188us (806us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.504469:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1003 00010000:00010000:2.0:1788572543.504654:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.504662:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572543.504870:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970461 00000001:00080000:2.0:1788572543.505090:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970462 00000001:00080000:2.0:1788572543.505139:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970463 00000100:00100000:2.0:1788572543.505221:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382361088:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572543.505274:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572543.505303:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.506441:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382361088 00000100:00100000:1.0:1788572543.506460:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382361088 00000100:00100000:1.0:1788572543.506555:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:1.0:1788572543.506566:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382361088:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572543.506671:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382361088:12345-0@lo:1000:mdt00_004.0 Request processed in 108us (1396us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.506679:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:2.0:1788572543.506742:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967591 gen 2 00000004:00080000:2.0:1788572543.506749:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967589 committed transno 4294967591 00000004:00080000:2.0:1788572543.506767:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967591 committed transno 4294967591 00000100:00100000:2.0:1788572543.506784:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382361088:0@lo:1000:mdt00_004.0 00000001:00080000:2.0:1788572543.507212:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970464 00000100:00100000:2.0:1788572543.507285:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382361216:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572543.507322:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.507778:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382361216 00000100:00100000:0.0:1788572543.507786:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382361216 00000100:00100000:0.0:1788572543.507820:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1788572543.507829:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382361216:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572543.508063:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382361216:12345-0@lo:1000:mdt00_004.0 Request processed in 234us (739us total) trans 4294967595 rc 0/0 00000100:00100000:0.0:1788572543.508070:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:2.0:1788572543.508407:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb900 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.508436:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382361216:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1788572543.508487:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508499:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508508:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508515:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.508520:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508534:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508551:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:2.0:1788572543.508554:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572543.508560:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508567:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162c8a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x88:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508597:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508604:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508611:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508617:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.508620:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508627:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508635:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:2.0:1788572543.508638:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572543.508641:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508652:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935000/0x85ace67feb15f444 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x82:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508674:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508687:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508698:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508705:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508712:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508718:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508723:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.508725:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508731:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508737:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:2.0:1788572543.508739:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.508741:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508747:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c91 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x82:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508756:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508761:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508766:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508771:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.508774:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508780:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508787:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:2.0:1788572543.508789:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572543.508792:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.508798:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162c83 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x82:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.509348:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35917:x1875453360395520:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 9520us (10105us total) trans 4294970465 rc 0/0 00000100:00100000:2.0:1788572543.509359:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4209 00000020:00080000:2.0:1788572543.509365:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:2.0:1788572543.511998:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572543.512040:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572543.512089:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 142 count 101 00000040:00080000:2.0:1788572543.512097:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:2.0:1788572543.512276:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382361344:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572543.512364:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.518184:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.519090:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360395904 00000100:00100000:2.0:1788572543.519103:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360395904 00000100:00100000:2.0:1788572543.519162:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 907 00000100:00100000:2.0:1788572543.519172:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35917:x1875453360395904:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572543.519618:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x88:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572543.519661:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970466 00010000:00010000:2.0:1788572543.519707:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.519723:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.519736:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.519745:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.519752:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.519759:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.519845:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572543.520041:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35917:x1875453360395904:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 869us (1866us total) trans 4294970467 rc 0/0 00000100:00100000:2.0:1788572543.520050:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 907 00000020:00080000:2.0:1788572543.520056:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788572543.520765:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.520791:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.520809:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.520889:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382361344:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572543.524474:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396032 00000100:00100000:1.0:1788572543.524487:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396032 00000100:00100000:1.0:1788572543.524553:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396160 00000100:00100000:1.0:1788572543.524558:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396160 00000100:00100000:2.0:1788572543.524624:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396288 00000100:00100000:2.0:1788572543.524629:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396288 00000100:00100000:2.0:1788572543.524665:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4210 00000100:00100000:2.0:1788572543.524675:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35922:x1875453360396032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572543.524706:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb1:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572543.524834:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35922:x1875453360396032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 160us (4071us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.524843:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4210 00000100:00100000:2.0:1788572543.524875:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4211 00000100:00100000:2.0:1788572543.524882:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35920:x1875453360396160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572543.524894:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xaf:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572543.524951:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35920:x1875453360396160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 70us (4158us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.524959:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4211 00000100:00100000:2.0:1788572543.524972:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4212 00000100:00100000:2.0:1788572543.524979:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35921:x1875453360396288:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572543.524992:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xae:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572543.525055:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35921:x1875453360396288:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 77us (4244us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.525062:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4212 00010000:00010000:0.0:1788572543.525346:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572543.525356:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.525366:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572543.525371:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.525381:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.525398:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x88:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572543.525401:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572543.525406:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.525412:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162c98 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x88:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.525432:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.525434:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572543.525435:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572543.533068:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.533116:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.533196:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396416 00000100:00100000:3.0:1788572543.533217:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396416 00000100:00100000:3.0:1788572543.533272:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396544 00000100:00100000:3.0:1788572543.533275:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396544 00000100:00100000:3.0:1788572543.533293:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4213 00000100:00100000:3.0:1788572543.533302:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35922:x1875453360396416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.533390:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533411:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533422:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533437:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533445:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533462:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533491:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533500:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533507:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572543.533551:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb1:0x0]: returning size 3 00000004:00200000:3.0:1788572543.533555:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572543.533570:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533582:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533589:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533595:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533601:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.533605:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533612:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533625:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572543.533628:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.533632:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533638:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162c9f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533651:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533656:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533663:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533667:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.533669:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533675:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533682:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3425802f (ffff9627ffc51300) 00010000:00010000:3.0:1788572543.533684:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.533687:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533697:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb162ca6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3425802f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533712:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533718:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533722:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533727:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.533730:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533735:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533742:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:3.0:1788572543.533745:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.533747:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533753:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162cad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.533827:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35922:x1875453360396416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 524us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.533834:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4213 00000100:00100000:3.0:1788572543.533866:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4214 00000100:00100000:3.0:1788572543.533874:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35920:x1875453360396544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.533908:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533916:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533923:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533934:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533940:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533947:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533964:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533972:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.533978:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572543.533999:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaf:0x0]: returning size 3 00000004:00200000:3.0:1788572543.534031:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572543.534039:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534048:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534054:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534060:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534066:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.534068:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534075:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534083:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572543.534085:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.534088:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534093:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162cb4 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534102:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534108:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534113:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534119:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.534122:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534127:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534134:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x76dad380 (ffff9627ffc51500) 00010000:00010000:3.0:1788572543.534136:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.534139:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534145:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162cbb lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x76dad380 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534157:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534163:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534169:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534174:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.534176:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534182:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534193:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:3.0:1788572543.534195:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.534197:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.534203:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb162cc2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.534234:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35920:x1875453360396544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 360us (1117us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.534240:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4214 00000100:00100000:0.0:1788572543.534901:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.534966:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396672 00000100:00100000:1.0:1788572543.534972:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396672 00000100:00100000:1.0:1788572543.535036:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4215 00000100:00100000:1.0:1788572543.535045:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35921:x1875453360396672:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.535084:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535096:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535105:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535120:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535126:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535132:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535151:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535159:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535166:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572543.535191:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xae:0x0]: returning size 3 00000004:00200000:1.0:1788572543.535195:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572543.535807:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535822:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535828:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535833:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.535839:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.535842:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536108:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536120:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572543.536123:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.536128:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536135:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb162cc9 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536155:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536161:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536166:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536172:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.536175:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536181:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536188:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x313ff2f4 (ffff9627d1112e00) 00010000:00010000:1.0:1788572543.536190:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.536192:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536198:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb162cd0 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x313ff2f4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536226:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536232:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536237:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536242:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.536245:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536249:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536257:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627c5716600) 00010000:00010000:1.0:1788572543.536259:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.536261:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.536266:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162cd7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.536341:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35921:x1875453360396672:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1295us (1441us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.536350:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4215 00000100:00100000:0.0:1788572543.540442:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.540694:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360396928 00000100:00100000:3.0:1788572543.540704:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360396928 00000100:00100000:3.0:1788572543.540771:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4216 00000100:00100000:3.0:1788572543.540781:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35922:x1875453360396928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.540804:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.540827:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162cde lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc23 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541652:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541670:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541682:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572543.541696:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb1:0x0]: returning size 3 00010000:00010000:2.0:1788572543.541807:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541814:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541821:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541826:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.541831:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541838:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541904:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:2.0:1788572543.541908:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572543.541913:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.541919:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162ce5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.542324:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162cde lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc23 expref: 82 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.542337:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572543.542339:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572543.542343:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162cde lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc23 expref: 82 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.542354:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd400, rc 0) 00000100:00100000:2.0:1788572543.542472:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35922:x1875453360396928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1692us (2038us total) trans 4294970468 rc 301/301 00000100:00100000:2.0:1788572543.542481:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4216 00000100:00100000:1.0:1788572543.551763:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.551895:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360397568 00000100:00100000:2.0:1788572543.551905:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360397568 00000100:00100000:2.0:1788572543.551978:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4217 00000100:00100000:2.0:1788572543.551989:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35922:x1875453360397568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572543.552048:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb1:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572543.552177:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35922:x1875453360397568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 189us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.552186:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4217 00000100:00100000:2.0:1788572543.555543:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.555591:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.555680:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360397440 00000100:00100000:0.0:1788572543.555690:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360397440 00000100:00100000:0.0:1788572543.555752:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360397184 00000100:00100000:0.0:1788572543.555755:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360397184 00000100:00100000:0.0:1788572543.555777:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4218 00000100:00100000:0.0:1788572543.555788:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35920:x1875453360397440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.555809:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.555830:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162cf3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc31 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.556039:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.556055:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.556068:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.556080:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaf:0x0]: returning size 3 00010000:00010000:0.0:1788572543.556176:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.556185:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.556192:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.556198:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00000100:00100000:3.0:1788572543.556333:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4219 00000100:00100000:3.0:1788572543.556345:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35921:x1875453360397184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.556354:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.556364:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162d08 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc2a expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556412:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556421:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556429:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572543.556437:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xae:0x0]: returning size 3 00010000:00010000:3.0:1788572543.556493:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556500:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556506:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556512:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.556515:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556523:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556537:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627c5716600) 00010000:00010000:3.0:1788572543.556540:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572543.556545:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556552:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d0f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556600:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162d08 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc2a expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556611:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572543.556613:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572543.556617:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162d08 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc2a expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.556627:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc 0) 00000100:00100000:3.0:1788572543.556688:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35921:x1875453360397184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 345us (1095us total) trans 4294970470 rc 301/301 00000100:00100000:3.0:1788572543.556696:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4219 00010000:00010000:0.0:1788572543.557201:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.557269:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.557280:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:0.0:1788572543.557282:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572543.557538:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.557546:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162cfa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.557590:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162cf3 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc31 expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.557599:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572543.557601:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572543.557605:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162cf3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc31 expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.557621:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572543.557665:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35920:x1875453360397440:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1879us (2134us total) trans 4294970469 rc 301/301 00000100:00100000:0.0:1788572543.557672:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4218 00000100:00100000:2.0:1788572543.563870:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.564522:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360397696 00000100:00100000:3.0:1788572543.564534:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360397696 00000100:00100000:2.0:1788572543.564540:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.564589:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.567617:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360397824 00000100:00100000:3.0:1788572543.567633:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360397824 00000100:00100000:3.0:1788572543.567690:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360397952 00000100:00100000:3.0:1788572543.567694:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360397952 00000100:00100000:3.0:1788572543.567721:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4220 00000100:00100000:3.0:1788572543.567732:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35922:x1875453360397696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.567754:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.567776:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162d1d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc38 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.567846:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.567865:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.567877:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.567892:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.567899:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.567905:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568441:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162d32 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568462:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162d32 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568470:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162d32 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572543.568499:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:3.0:1788572543.568503:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572543.568527:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162d32 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568539:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568551:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568558:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568564:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.568568:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568575:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568588:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572543.568591:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.568597:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568603:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162d24 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568615:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568621:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568627:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568631:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.568635:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568642:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568649:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc51500) 00010000:00010000:3.0:1788572543.568652:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.568654:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568660:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162d2b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568695:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb162d1d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc38 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568709:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162d32 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc38 expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.568717:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572543.568719:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572543.568722:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc000, rc 0) 00000100:00100000:3.0:1788572543.568789:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35922:x1875453360397696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1058us (4940us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.568797:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4220 00000100:00100000:3.0:1788572543.571509:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4221 00000100:00100000:3.0:1788572543.571532:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35920:x1875453360397824:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572543.571569:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xaf:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572543.571686:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35920:x1875453360397824:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 164us (7145us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.571695:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4221 00000100:00100000:3.0:1788572543.571718:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4222 00000100:00100000:3.0:1788572543.571727:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35921:x1875453360397952:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572543.571745:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xae:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572543.571787:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35921:x1875453360397952:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 62us (7198us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.571794:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4222 00000100:00100000:3.0:1788572543.572808:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.572883:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360398208 00000100:00100000:1.0:1788572543.572892:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360398208 00000100:00100000:1.0:1788572543.572944:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4223 00000100:00100000:1.0:1788572543.572953:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35922:x1875453360398208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.572966:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.572970:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572543.572997:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb162d32 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc38 expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.573052:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.573059:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.573073:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d39 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc3f expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572543.573861:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4c:0x0], index 1: 0 00010000:00010000:1.0:1788572543.576894:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162d40 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.576918:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162d40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.576942:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162d40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.577042:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d39 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc3f expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.577062:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162d40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc3f expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.577071:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:1.0:1788572543.577074:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572543.577078:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97e00, rc 0) 00000100:00100000:1.0:1788572543.578185:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35922:x1875453360398208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5228us (5376us total) trans 4294970471 rc 0/0 00000100:00100000:1.0:1788572543.578197:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4223 00000100:00100000:3.0:1788572543.578426:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.580656:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360398336 00000100:00100000:2.0:1788572543.580672:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360398336 00000100:00100000:2.0:1788572543.580907:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4224 00000100:00100000:2.0:1788572543.580919:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35920:x1875453360398336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.580955:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.580977:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d4e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc46 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581088:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581105:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581117:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581133:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581140:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581147:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581595:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162d63 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581606:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162d63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581613:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162d63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572543.581642:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:2.0:1788572543.581647:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572543.581667:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162d63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581680:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581686:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581693:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581700:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.581704:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581712:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581724:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572543.581726:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.581731:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581742:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162d55 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581754:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581760:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581766:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581770:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.581773:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581778:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581785:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627e7a95e00) 00010000:00010000:2.0:1788572543.581787:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.581790:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581796:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162d5c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581830:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d4e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc46 expref: 87 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581848:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162d63 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc46 expref: 86 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.581855:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572543.581857:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572543.581859:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05400, rc 0) 00000100:00100000:2.0:1788572543.581973:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35920:x1875453360398336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1056us (3546us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.581981:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4224 00000100:00100000:3.0:1788572543.586668:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.586892:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360398592 00000100:00100000:1.0:1788572543.586902:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360398592 00000100:00100000:1.0:1788572543.586963:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4225 00000100:00100000:1.0:1788572543.586972:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35922:x1875453360398592:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.587062:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572543.587089:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162d40 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc3f expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587109:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627d1112e00) 00010000:00010000:1.0:1788572543.587126:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162d63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc46 expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587134:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.587138:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.587161:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587174:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587184:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587199:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587216:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587224:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587460:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587469:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587476:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587482:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.587487:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587495:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587504:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572543.587507:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.587510:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587516:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162d6a lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587525:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587531:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587538:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587543:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.587546:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587551:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587558:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xa46c166d (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.587560:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.587563:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.587568:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d71 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0xa46c166d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.587671:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35922:x1875453360398592:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 698us (1011us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572543.587679:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4225 00000100:00100000:3.0:1788572543.599799:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.599923:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360398720 00000100:00100000:0.0:1788572543.599932:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360398720 00000100:00100000:0.0:1788572543.599998:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4226 00000100:00100000:0.0:1788572543.600048:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35921:x1875453360398720:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.600076:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.600102:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162d78 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc4d expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.600174:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.600188:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.600208:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.600223:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.600229:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.600235:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601300:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d8d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601317:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601326:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572543.601504:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:2.0:1788572543.601509:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572543.601532:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601546:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601554:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601565:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601571:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.601575:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601583:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601596:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572543.601599:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.601605:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601611:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162d7f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601630:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601636:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601642:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601647:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.601651:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601660:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601668:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627e7451700) 00010000:00010000:2.0:1788572543.601670:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.601674:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601679:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162d86 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601725:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162d78 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc4d expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601742:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d8d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc4d expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.601750:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572543.601753:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572543.601756:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572543.601920:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35921:x1875453360398720:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1874us (2133us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.601930:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4226 00000100:00100000:3.0:1788572543.609878:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.610309:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360398848 00000100:00100000:1.0:1788572543.610319:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360398848 00000100:00100000:1.0:1788572543.610380:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4227 00000100:00100000:1.0:1788572543.610577:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35922:x1875453360398848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.610601:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.610624:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d94 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc54 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.611489:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.611506:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.611518:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.611632:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162d94 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.612117:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.612127:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:0.0:1788572543.612131:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572543.612135:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:0.0:1788572543.612282:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35922:x1875453360398848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1888us (2408us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.612354:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4227 00000100:00100000:1.0:1788572543.621437:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.621754:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.622570:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360399104 00000100:00100000:0.0:1788572543.622580:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360399104 00000100:00100000:0.0:1788572543.622751:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360399360 00000100:00100000:0.0:1788572543.622755:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360399360 00000100:00100000:0.0:1788572543.622787:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4228 00000100:00100000:0.0:1788572543.622796:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35920:x1875453360399104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.622816:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.622837:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162da2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc5b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572543.623564:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4d:0x0], index 0: 0 00010000:00010000:0.0:1788572543.623829:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162da9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.623856:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162da9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.623869:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162da9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.624053:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4229 00000100:00100000:1.0:1788572543.624065:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35922:x1875453360399360:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.624119:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.624129:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.624137:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572543.624689:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].62, 88 off13240 00000004:00010000:1.0:1788572543.624768:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002a029e49 reply state 00000000895ab237 transno 4294970473 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.624777:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572543.624882:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35922:x1875453360399360:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 818us (3126us total) trans 4294970473 rc 0/0 00000100:00100000:1.0:1788572543.624891:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4229 00010000:00010000:0.0:1788572543.627174:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162da2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc5b expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.627733:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162da9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc5b expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.627745:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451e00) 00010000:00010000:0.0:1788572543.627748:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572543.627752:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572543.628179:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35920:x1875453360399104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5381us (6747us total) trans 4294970472 rc 0/0 00000100:00100000:0.0:1788572543.628189:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4228 00000100:00100000:1.0:1788572543.628617:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572543.628724:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.628734:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.628741:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.628747:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.628752:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.628761:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.628776:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:0.0:1788572543.628779:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572543.628784:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.628790:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162db7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.629877:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360399488 00000100:00100000:1.0:1788572543.629889:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360399488 00000100:00100000:0.0:1788572543.630562:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4230 00000100:00100000:0.0:1788572543.630573:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35922:x1875453360399488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.630595:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.630616:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc62 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.630688:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.630703:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.630723:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:0.0:1788572543.630726:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572543.630729:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572543.630794:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35922:x1875453360399488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 223us (2179us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.630801:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4230 00000100:00100000:1.0:1788572543.636812:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.637185:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360399744 00000100:00100000:2.0:1788572543.637194:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360399744 00000100:00100000:2.0:1788572543.637260:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4231 00000100:00100000:2.0:1788572543.637271:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35920:x1875453360399744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.637384:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572543.637410:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162da9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc5b expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637431:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451e00) 00010000:00010000:2.0:1788572543.637446:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162d8d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc4d expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637465:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572543.637469:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572543.637497:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637511:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637521:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637535:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637543:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.637550:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638299:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638313:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638320:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638326:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.638330:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638338:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638348:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572543.638351:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.638355:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638361:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162dc5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638370:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638377:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638383:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638388:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.638392:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638402:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638409:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x503d8860 (ffff9627e7a95e00) 00010000:00010000:2.0:1788572543.638412:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.638415:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.638420:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162dcc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x503d8860 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.638519:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35920:x1875453360399744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1249us (1715us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572543.638527:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4231 00000100:00100000:1.0:1788572543.643745:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.643788:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.643931:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360399872 00000100:00100000:0.0:1788572543.643943:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360399872 00000100:00100000:0.0:1788572543.644034:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360400128 00000100:00100000:0.0:1788572543.644039:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360400128 00000100:00100000:0.0:1788572543.644063:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4232 00000100:00100000:0.0:1788572543.644073:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35920:x1875453360399872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.644096:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.644122:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162dd3 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc69 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.644158:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.644172:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.644182:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.644286:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162dd3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.644318:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.644327:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572543.644329:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572543.644333:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:0.0:1788572543.644387:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35920:x1875453360399872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 315us (650us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.644394:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4232 00000100:00100000:0.0:1788572543.644414:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4233 00000100:00100000:0.0:1788572543.644421:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35921:x1875453360400128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.644429:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.644438:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162de1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc70 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572543.645113:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4d:0x0], index 1: 0 00000100:00100000:1.0:1788572543.645815:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.645988:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360400256 00000100:00100000:2.0:1788572543.645996:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360400256 00000100:00100000:2.0:1788572543.646056:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1131 00000100:00100000:2.0:1788572543.646063:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360400256:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572543.646254:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360400256:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 189us (441us total) trans 4294967596 rc 0/0 00000100:00100000:2.0:1788572543.646261:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1131 00010000:00010000:0.0:1788572543.646552:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162de8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.646566:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162de8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.646576:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162de8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.647477:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162de1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc70 expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.647504:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162de8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc70 expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.647513:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451000) 00010000:00010000:0.0:1788572543.647516:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572543.647520:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:0.0:1788572543.647556:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35921:x1875453360400128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3135us (3767us total) trans 4294970474 rc 0/0 00000100:00100000:0.0:1788572543.647563:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4233 00000100:00100000:1.0:1788572543.652242:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.652306:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.652499:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360400512 00000100:00100000:2.0:1788572543.652508:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360400512 00000100:00100000:2.0:1788572543.652556:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360400640 00000100:00100000:2.0:1788572543.652558:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360400640 00000100:00100000:2.0:1788572543.652582:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4234 00000100:00100000:2.0:1788572543.652592:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35920:x1875453360400512:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.652702:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.652719:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.652730:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.654120:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4235 00000100:00100000:3.0:1788572543.654138:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35921:x1875453360400640:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.654156:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.654173:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162dfd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc77 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.654256:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.654269:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.654279:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.654836:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.654847:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.654855:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572543.655620:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].65, 88 off13432 00000004:00010000:2.0:1788572543.655754:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000054b3cb2a reply state 000000004c384c75 transno 4294970475 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.655767:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572543.655901:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35920:x1875453360400512:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 3305us (3664us total) trans 4294970475 rc 0/0 00000100:00100000:2.0:1788572543.655910:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4234 00010000:00010000:3.0:1788572543.656401:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162e12 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656415:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162e12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656423:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162e12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572543.656447:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:3.0:1788572543.656462:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572543.656478:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162e12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656491:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656497:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656504:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656514:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.656519:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656525:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656539:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572543.656542:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.656548:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656555:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162e04 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656570:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656576:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656581:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656586:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.656588:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656593:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656604:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc51500) 00010000:00010000:3.0:1788572543.656606:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572543.656609:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.656615:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162e0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.657581:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162dfd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc77 expref: 95 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.657602:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162e12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc77 expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.657610:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572543.657612:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572543.657616:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572543.657878:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35921:x1875453360400640:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3746us (5569us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.657887:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4235 00000100:00100000:0.0:1788572543.659503:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.659865:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360400896 00000100:00100000:2.0:1788572543.659874:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360400896 00000100:00100000:2.0:1788572543.659973:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 908 00000100:00100000:2.0:1788572543.659985:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35922:x1875453360400896:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.660506:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.660518:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.660525:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.660531:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.660536:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.660544:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.660559:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572543.660561:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572543.660566:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.660572:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162df6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.660874:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35922:x1875453360400896:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 890us (1377us total) trans 4294970476 rc 0/0 00000100:00100000:2.0:1788572543.660882:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 908 00000100:00100000:1.0:1788572543.663690:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.663829:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360401024 00000100:00100000:0.0:1788572543.663838:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360401024 00000100:00100000:0.0:1788572543.663906:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4236 00000100:00100000:0.0:1788572543.663916:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35920:x1875453360401024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.663950:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.663972:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc7e expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.664076:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.664091:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.664113:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572543.664116:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572543.664120:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d600, rc 0) 00000100:00100000:0.0:1788572543.664169:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35920:x1875453360401024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 255us (488us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.664176:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4236 00000100:00100000:1.0:1788572543.668261:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.668525:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360401152 00000100:00100000:0.0:1788572543.668591:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360401152 00000100:00100000:0.0:1788572543.668692:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1132 00000100:00100000:0.0:1788572543.668703:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360401152:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.669274:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360401152:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 571us (1024us total) trans 4294967599 rc 0/0 00000100:00100000:0.0:1788572543.669284:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1132 00000100:00100000:1.0:1788572543.672482:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.672585:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.673415:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360401408 00000100:00100000:2.0:1788572543.673427:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360401408 00000100:00100000:2.0:1788572543.673537:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360401664 00000100:00100000:2.0:1788572543.673542:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360401664 00000100:00100000:0.0:1788572543.673580:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4237 00000100:00100000:0.0:1788572543.673591:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35922:x1875453360401408:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.673723:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35922:x1875453360401408:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 133us (1249us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.673730:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4237 00000100:00100000:0.0:1788572543.673751:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4238 00000100:00100000:0.0:1788572543.673758:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35921:x1875453360401664:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.673789:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788572543.673814:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162de8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc70 expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673838:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451000) 00010000:00010000:0.0:1788572543.673858:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162e12 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc77 expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673869:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572543.673873:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572543.673896:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673911:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673931:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673950:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673957:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.673963:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674276:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674284:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674791:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674798:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.674803:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674812:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674827:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572543.674829:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.674833:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674839:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162e20 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674853:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674859:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674864:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674870:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.674873:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674878:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674886:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x172b14cd (ffff9627e7451b00) 00010000:00010000:0.0:1788572543.674888:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572543.674891:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.674897:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb162e27 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x172b14cd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.674975:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35921:x1875453360401664:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1216us (2389us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572543.674983:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4238 00000100:00100000:3.0:1788572543.676390:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.676491:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360401920 00000100:00100000:2.0:1788572543.676500:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360401920 00000100:00100000:2.0:1788572543.676573:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 909 00000100:00100000:2.0:1788572543.676583:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35920:x1875453360401920:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.677262:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.677376:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360402048 00000100:00100000:0.0:1788572543.677382:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360402048 00000100:00100000:0.0:1788572543.677420:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4239 00000100:00100000:0.0:1788572543.677427:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35921:x1875453360402048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.677439:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.677469:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e2e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc85 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.677504:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.677520:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.677530:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.677616:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e2e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 97 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.677709:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 96 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.677719:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:0.0:1788572543.677721:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572543.677724:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:0.0:1788572543.677779:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35921:x1875453360402048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 352us (519us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.677787:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4239 00000100:00100000:2.0:1788572543.678585:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35920:x1875453360401920:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 2000us (2200us total) trans 4294970477 rc 0/0 00000100:00100000:2.0:1788572543.678594:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 909 00000100:00100000:1.0:1788572543.678917:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.679316:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360402176 00000100:00100000:1.0:1788572543.679323:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360402176 00000100:00100000:1.0:1788572543.679368:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4240 00000100:00100000:1.0:1788572543.679378:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35922:x1875453360402176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.679391:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.679406:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162e3c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc8c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.679494:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162e43 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.679507:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162e43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.679517:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162e43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572543.679545:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572543.679561:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162e43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.679583:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb162e3c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc8c expref: 97 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.679597:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162e43 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc8c expref: 96 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.679609:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.679612:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572543.679615:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:1.0:1788572543.679786:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35922:x1875453360402176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 408us (868us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.679793:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4240 00000100:00100000:1.0:1788572543.680414:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.680478:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360402432 00000100:00100000:0.0:1788572543.680483:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360402432 00000100:00100000:0.0:1788572543.680523:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4241 00000100:00100000:0.0:1788572543.680531:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35920:x1875453360402432:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.680605:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35920:x1875453360402432:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 74us (191us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.680611:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4241 00000100:00100000:1.0:1788572543.681974:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.682231:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360402816 00000100:00100000:0.0:1788572543.682239:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360402816 00000100:00100000:0.0:1788572543.682284:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4242 00000100:00100000:0.0:1788572543.682302:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35921:x1875453360402816:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.682401:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.682417:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.682427:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.682972:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.683130:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360402560 00000100:00100000:3.0:1788572543.683135:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360402560 00000100:00100000:3.0:1788572543.683172:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1133 00000100:00100000:3.0:1788572543.683180:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360402560:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.683192:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572543.683207:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc93 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572543.683232:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737f, blocks: 0x1 00010000:00010000:3.0:1788572543.683247:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=891s v=(0 0) 00010000:00010000:3.0:1788572543.683251:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572543.683259:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 12 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.683274:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 12 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.683298:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc400, rc 0) 00000100:00100000:3.0:1788572543.683718:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360402560:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 539us (746us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.683724:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1133 00000040:00080000:0.0:1788572543.683824:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].63, 88 off13328 00000004:00080000:0.0:1788572543.683833:0:5837:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 891752827919ns, next at 892752824907ns, rc = 0 00000004:00010000:0.0:1788572543.683921:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003ba5e1fe reply state 000000003e91e0f7 transno 4294970478 ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.683930:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572543.683983:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35921:x1875453360402816:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1684us (2015us total) trans 4294970478 rc 0/0 00000100:00100000:0.0:1788572543.683988:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4242 00000100:00100000:1.0:1788572543.684514:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.684569:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360402944 00000100:00100000:0.0:1788572543.684575:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360402944 00000100:00100000:0.0:1788572543.684726:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4243 00000100:00100000:0.0:1788572543.684736:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35920:x1875453360402944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.684749:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.684762:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162e58 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efc9a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.684820:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162e5f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.684829:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162e5f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.684836:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162e5f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.684857:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.684872:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162e5f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.684899:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162e58 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc9a expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.684916:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162e5f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc9a expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.684940:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572543.684943:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572543.684947:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:0.0:1788572543.684969:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35920:x1875453360402944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 236us (458us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.684974:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4243 00000100:00100000:1.0:1788572543.687195:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.687574:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360403072 00000100:00100000:2.0:1788572543.687584:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360403072 00000100:00100000:2.0:1788572543.687656:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4244 00000100:00100000:2.0:1788572543.687666:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35921:x1875453360403072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.687689:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572543.687707:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efca1 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.687760:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.687774:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.687791:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:2.0:1788572543.687795:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572543.687798:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572543.687857:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35921:x1875453360403072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 192us (671us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.687867:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4244 00010000:00010000:0.0:1788572543.688825:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.688837:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.688844:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.688850:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572543.688854:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.688862:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.688876:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:0.0:1788572543.688879:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572543.688884:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.688891:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e4a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.692264:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.692496:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.692591:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360403456 00000100:00100000:1.0:1788572543.692613:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360403456 00000100:00100000:1.0:1788572543.692682:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4245 00000100:00100000:1.0:1788572543.692693:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35922:x1875453360403456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.692821:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35922:x1875453360403456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 130us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.692828:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4245 00000100:00100000:3.0:1788572543.696710:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.698922:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.702773:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360403200 00000100:00100000:0.0:1788572543.702792:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360403200 00000100:00100000:0.0:1788572543.702950:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360403584 00000100:00100000:0.0:1788572543.702955:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360403584 00000100:00100000:1.0:1788572543.703586:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360403712 00000100:00100000:1.0:1788572543.703592:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360403712 00000100:00100000:1.0:1788572543.703652:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1134 00000100:00100000:1.0:1788572543.703664:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360403200:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.703682:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.703703:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efca8 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572543.703733:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737f, blocks: 0x1 00010000:00010000:1.0:1788572543.703754:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=891s v=(0 0) 00010000:00010000:1.0:1788572543.703758:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.703768:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.703785:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.703797:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97e00, rc 0) 00000100:00100000:1.0:1788572543.703860:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360403200:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 198us (11606us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.703867:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1134 00000100:00100000:1.0:1788572543.703891:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1135 00000100:00100000:1.0:1788572543.703898:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360403584:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.704385:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360403584:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 485us (7684us total) trans 4294967597 rc 0/0 00000100:00100000:1.0:1788572543.704393:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1135 00000100:00100000:1.0:1788572543.704408:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1136 00000100:00100000:1.0:1788572543.704414:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35922:x1875453360403712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.704422:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.704432:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcaf expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.704469:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.704478:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.704482:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.704487:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.704518:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.704532:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.704546:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382361472:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572543.704611:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.726395:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.726441:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.726616:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360403968 00000100:00100000:0.0:1788572543.726627:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360403968 00000100:00100000:0.0:1788572543.726696:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4246 00000100:00100000:0.0:1788572543.726706:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35920:x1875453360403968:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.726829:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35920:x1875453360403968:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 125us (443us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.726837:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4246 00000100:00100000:1.0:1788572543.727257:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360404224 00000100:00100000:1.0:1788572543.727264:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360404224 00000100:00100000:1.0:1788572543.727314:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 910 00000100:00100000:1.0:1788572543.727322:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35921:x1875453360404224:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572543.728189:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.728784:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35921:x1875453360404224:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1010us (1890us total) trans 4294970479 rc 0/0 00000100:00100000:1.0:1788572543.728797:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 910 00000100:00100000:1.0:1788572543.728855:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382361472:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.728888:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.728910:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96a00, rc 0) 00000100:00100000:1.0:1788572543.728936:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35922:x1875453360403712:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 24522us (30024us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.728943:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1136 00000100:00100000:0.0:1788572543.729250:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360404352 00000100:00100000:0.0:1788572543.729257:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360404352 00010000:00010000:0.0:1788572543.729294:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.729327:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1004 00000100:00100000:0.0:1788572543.729346:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360404352:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.729359:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.729379:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.729393:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.729414:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cbe4dd9a/0x85ace67feb162e51 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc93 expref: 14 pid: 13272 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.729428:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.729433:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.729441:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.729461:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.729595:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.729611:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382361600:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572543.729734:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.747448:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.747518:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.747586:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.747690:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360404608 00000100:00100000:2.0:1788572543.747701:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360404608 00000100:00100000:2.0:1788572543.747767:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4247 00000100:00100000:2.0:1788572543.747778:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35921:x1875453360404608:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572543.747888:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35921:x1875453360404608:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 112us (458us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.747896:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4247 00000100:00100000:0.0:1788572543.748205:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382361600:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572543.748229:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.748253:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360404352:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 18918us (20066us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.748262:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1004 00000100:00100000:1.0:1788572543.748880:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360404864 00000100:00100000:1.0:1788572543.748888:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360404864 00002000:00100000:1.0:1788572543.748926:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4c:0x0] (0->2) req@ffff9627ff0aa680 x1875453360404864/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/0 e 0 to 0 dl 1788572554 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.748937:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff0aa680 with x1875453360404864 ext(0->2) 00002000:00010000:1.0:1788572543.748942:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ff0aa680 00000100:00100000:1.0:1788572543.749251:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 257 00000100:00100000:1.0:1788572543.749262:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360404864:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.749827:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360404864:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 566us (2239us total) trans 4294967598 rc 0/0 00000100:00100000:1.0:1788572543.749836:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 257 00002000:00100000:1.0:1788572543.749843:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4c:0x0] (0->2) req@ffff9627ff0aa680 x1875453360404864/t4294967598(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/448 e 0 to 0 dl 1788572554 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.749852:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff0aa680 with x1875453360404864 ext(0->2) 00002000:00010000:1.0:1788572543.749856:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ff0aa680 00000100:00100000:0.0:1788572543.749933:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360404736 00000100:00100000:0.0:1788572543.749939:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360404736 00000100:00100000:0.0:1788572543.749970:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1137 00000100:00100000:0.0:1788572543.749977:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35920:x1875453360404736:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.749987:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.750232:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcb6 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.750261:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.750270:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.750275:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.750280:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.750313:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.750328:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.750344:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382361728:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.750397:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.760307:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.760377:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.760414:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.760556:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382361728:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.760596:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.760631:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:0.0:1788572543.760700:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35920:x1875453360404736:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 10722us (13182us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.760709:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1137 00000100:00100000:1.0:1788572543.760965:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360404992 00000100:00100000:1.0:1788572543.760974:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360404992 00000100:00100000:1.0:1788572543.761059:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360405248 00000100:00100000:1.0:1788572543.761064:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360405248 00000100:00100000:1.0:1788572543.761085:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4248 00000100:00100000:1.0:1788572543.761094:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35922:x1875453360404992:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.761352:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4249 00000100:00100000:0.0:1788572543.761361:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35921:x1875453360405248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.761372:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.761395:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e82 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcbd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.761482:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e89 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.761498:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.761515:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572543.761549:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572543.761565:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.761593:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162e82 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcbd expref: 103 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.761622:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e89 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcbd expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.761630:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:0.0:1788572543.761633:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572543.761792:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572543.761832:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35921:x1875453360405248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 472us (1416us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.761838:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4249 00000100:00100000:2.0:1788572543.761962:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360405120 00000100:00100000:2.0:1788572543.761968:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360405120 00010000:00010000:2.0:1788572543.761994:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.762064:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1005 00000100:00100000:2.0:1788572543.762072:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360405120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572543.762083:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572543.762103:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.762116:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.762134:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb162e6d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca8 expref: 14 pid: 10416 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.762148:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572543.762152:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572543.762161:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.762171:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.762208:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.762224:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382361856:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572543.762277:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13170c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.771437:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.771552:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382361856:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572543.771577:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572543.771643:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360405120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 9565us (11259us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.771652:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1005 00000100:00100000:3.0:1788572543.771909:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.772030:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360405504 00000100:00100000:1.0:1788572543.772036:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360405504 00000100:00100000:1.0:1788572543.772076:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1138 00000100:00100000:1.0:1788572543.772082:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360405504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.772095:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572543.772110:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcc4 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572543.772134:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737f, blocks: 0x1 00010000:00010000:1.0:1788572543.772149:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=891s v=(0 0) 00010000:00010000:1.0:1788572543.772152:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.772158:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 16 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.772172:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 16 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.772185:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:1.0:1788572543.772206:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360405504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 124us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.772212:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1138 00000100:00100000:0.0:1788572543.773118:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360405376 00000100:00100000:0.0:1788572543.773127:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360405376 00002000:00100000:0.0:1788572543.773167:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4d:0x0] (0->2) req@ffff9627ff0a9f80 x1875453360405376/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/0 e 0 to 0 dl 1788572554 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572543.773178:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff0a9f80 with x1875453360405376 ext(0->2) 00002000:00010000:0.0:1788572543.773183:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ff0a9f80 00000100:00100000:0.0:1788572543.773202:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 258 00000100:00100000:0.0:1788572543.773221:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360405376:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.776107:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360405376:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 2873us (4669us total) trans 4294967600 rc 0/0 00000100:00100000:0.0:1788572543.776123:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 258 00002000:00100000:0.0:1788572543.776137:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4d:0x0] (0->2) req@ffff9627ff0a9f80 x1875453360405376/t4294967600(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/448 e 0 to 0 dl 1788572554 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572543.776147:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff0a9f80 with x1875453360405376 ext(0->2) 00002000:00010000:0.0:1788572543.776153:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ff0a9f80 00000100:00100000:3.0:1788572543.781801:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.782779:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360405760 00000100:00100000:1.0:1788572543.782792:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360405760 00000100:00100000:1.0:1788572543.782857:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4250 00000100:00100000:1.0:1788572543.782868:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35921:x1875453360405760:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.783118:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35921:x1875453360405760:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 252us (1327us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.783127:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4250 00000100:00100000:3.0:1788572543.787729:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.787763:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.787792:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406016 00000100:00100000:3.0:1788572543.787826:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406016 00000100:00100000:3.0:1788572543.787995:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4251 00000100:00100000:3.0:1788572543.788038:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35920:x1875453360406016:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.794992:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360405888 00000100:00100000:0.0:1788572543.795119:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360405888 00000100:00100000:0.0:1788572543.795230:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1139 00000100:00100000:0.0:1788572543.795242:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35921:x1875453360405888:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.795262:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572543.795284:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efccb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.795317:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 17 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.795327:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.795331:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572543.795398:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.795438:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 17 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.795491:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 17 pid: 6681 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.795506:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382361984:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.795806:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.807146:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572543.807180:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406144 00000100:00100000:1.0:1788572543.807188:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406144 00010000:00010000:1.0:1788572543.807226:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 18 pid: 6681 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.807264:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1006 00000100:00100000:1.0:1788572543.807274:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2347:x1875453360406144:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572543.807288:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.807315:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 18 pid: 6681 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.807701:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382361984:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.807746:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 18 pid: 6681 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.808435:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 18 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.808463:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094de00, rc 0) 00000100:00100000:2.0:1788572543.808529:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:35921:x1875453360405888:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 13289us (20808us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.808537:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1139 00010000:00010000:1.0:1788572543.808744:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb162e90 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcc4 expref: 17 pid: 6681 timeout: 991 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.808762:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.808766:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572543.808775:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.808785:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.808825:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.808841:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382362112:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572543.808895:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.814396:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.814951:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382362112:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572543.814979:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572543.815185:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2347:x1875453360406144:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7911us (8046us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.815198:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1006 00000100:00100000:1.0:1788572543.816373:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406272 00000100:00100000:1.0:1788572543.816385:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406272 00002000:00100000:1.0:1788572543.816437:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4d:0x0] (0->2) req@ffff9627e786d500 x1875453360406272/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/0 e 0 to 0 dl 1788572554 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.816540:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786d500 with x1875453360406272 ext(0->2) 00002000:00010000:1.0:1788572543.816546:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e786d500 00000100:00100000:1.0:1788572543.816591:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 259 00000100:00100000:1.0:1788572543.816601:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360406272:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572543.817803:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360406272:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 1197us (3409us total) trans 4294967599 rc 0/0 00000100:00100000:1.0:1788572543.817813:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 259 00002000:00100000:1.0:1788572543.817822:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4d:0x0] (0->2) req@ffff9627e786d500 x1875453360406272/t4294967599(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:204/0 lens 496/448 e 0 to 0 dl 1788572554 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572543.817831:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786d500 with x1875453360406272 ext(0->2) 00002000:00010000:1.0:1788572543.817836:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e786d500 00000100:00100000:3.0:1788572543.828266:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.828314:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406400 00000100:00100000:2.0:1788572543.828324:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406400 00000100:00100000:2.0:1788572543.828466:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4252 00000100:00100000:2.0:1788572543.828477:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35921:x1875453360406400:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572543.828601:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35921:x1875453360406400:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 126us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.828609:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4252 00000100:00100000:3.0:1788572543.834906:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.834985:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406528 00000100:00100000:0.0:1788572543.834995:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406528 00000100:00100000:2.0:1788572543.835896:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1140 00000100:00100000:2.0:1788572543.835909:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360406528:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1788572543.846728:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970478 is committed 00000004:00080000:2.0:1788572543.846740:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 37 committed 00000020:00080000:2.0:1788572543.846764:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d12b0a80 refcount 2 committed 0 result 0 batchid 116 00000020:00080000:2.0:1788572543.846769:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3c00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d12b00c0 00000020:00080000:2.0:1788572543.846773:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4bd5400 00000020:00080000:2.0:1788572543.846783:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 114 batchid updated 00000001:00080000:2.0:1788572543.846792:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970479 is committed 00000020:00080000:2.0:1788572543.846807:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2f64000 refcount 2 committed 0 result 0 batchid 115 00000020:00080000:2.0:1788572543.846810:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad200 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2f64a80 00000020:00080000:2.0:1788572543.846813:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff03d400 00000020:00080000:2.0:1788572543.846841:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627e7495540 refcount 2 committed 0 result 0 batchid 117 00000020:00080000:2.0:1788572543.846844:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ea8200 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627e7495840 00000020:00080000:2.0:1788572543.846848:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ea8400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627caa5e400 00000100:00100000:2.0:1788572543.849187:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35920:x1875453360406016:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 60634us (60875us total) trans 0 rc 0/0 00000020:00080000:0.0:1788572543.849312:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 117 committed batchid 114 00000100:00100000:2.0:1788572543.849391:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4251 00000020:00080000:0.0:1788572543.849740:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 114 00000100:00100000:1.0:1788572543.850307:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35922:x1875453360404992:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 89210us (90006us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572543.850318:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4248 00000100:00100000:2.0:1788572543.850986:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382362240:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572543.851068:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572543.851307:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382362240 00000100:00100000:0.0:1788572543.851315:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382362240 00000100:00100000:0.0:1788572543.851516:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1141 00000100:00100000:0.0:1788572543.851526:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382362240:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572543.851858:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382362368:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572543.851888:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572543.851917:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382362368 00000100:00100000:1.0:1788572543.851919:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382362496:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572543.851922:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382362368 00000100:00100000:3.0:1788572543.851949:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1142 00000100:00100000:3.0:1788572543.851956:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382362368:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572543.852099:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382362368:12345-0@lo:2:osp-syn-1-0.0 Request processed in 143us (209us total) trans 4294967601 rc 0/0 00000100:00100000:3.0:1788572543.852130:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1142 00000100:00100000:1.0:1788572543.852550:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382362624:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572543.852584:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1788572543.852637:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e786f800/2, rc 0, transno 4294967601 00000100:00100000:1.0:1788572543.852643:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382362368:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572543.852671:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572543.852696:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382362624 00000100:00100000:1.0:1788572543.852700:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382362624 00000100:00100000:1.0:1788572543.852722:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382362496 00000100:00100000:1.0:1788572543.852725:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382362496 00000100:00100000:1.0:1788572543.852738:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1143 00000100:00100000:1.0:1788572543.852744:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382362624:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1788572543.852834:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572543.852838:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb mtime from disk: 0 -> 1788572543 00002000:00010000:1.0:1788572543.852840:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572543.852842:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb ctime from disk: 1788572543 -> 1788572543 00002000:00010000:1.0:1788572543.852845:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572543.852875:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382362624:12345-0@lo:2:osp-syn-1-0.0 Request processed in 131us (290us total) trans 4294967602 rc 0/0 00000100:00100000:1.0:1788572543.852882:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1143 00000100:00100000:1.0:1788572543.852894:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1144 00000100:00100000:1.0:1788572543.852900:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382362496:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1788572543.853966:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572543.853972:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb mtime from disk: 0 -> 1788572543 00002000:00010000:1.0:1788572543.853975:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572543.853977:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb ctime from disk: 1788572543 -> 1788572543 00002000:00010000:1.0:1788572543.853979:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572543.854054:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382362496:12345-0@lo:2:osp-syn-1-0.0 Request processed in 1152us (1380us total) trans 4294967603 rc 0/0 00000100:00100000:1.0:1788572543.854062:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1144 00000004:00080000:1.0:1788572543.854108:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e786e300/2, rc 0, transno 4294967602 00000100:00100000:1.0:1788572543.854113:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382362624:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572543.854257:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382362240:12345-0@lo:2:osp-syn-1-0.0 Request processed in 2732us (3189us total) trans 4294967600 rc 0/0 00000100:00100000:0.0:1788572543.854265:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1141 00000004:00080000:3.0:1788572543.855047:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e786dc00/2, rc 0, transno 4294967603 00000100:00100000:3.0:1788572543.855055:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382362496:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1788572543.856140:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e786ca80/2, rc 0, transno 4294967600 00000100:00100000:2.0:1788572543.856149:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382362240:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572543.859168:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.859217:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.859294:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406784 00000100:00100000:0.0:1788572543.859303:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406784 00000100:00100000:0.0:1788572543.859356:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360406656 00000100:00100000:0.0:1788572543.859360:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360406656 00000100:00100000:0.0:1788572543.859386:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1145 00000100:00100000:0.0:1788572543.859395:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360406784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572543.860993:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1146 00000100:00100000:0.0:1788572543.861032:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360406656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572543.885534:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360406528:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 49616us (50626us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572543.885549:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1140 00000001:00080000:1.0:1788572543.889250:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967595 is committed 00000001:00080000:1.0:1788572543.889268:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967594 is committed 00000001:00080000:1.0:1788572543.889276:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967599 is committed 00000100:00100000:1.0:1788572543.892308:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.892395:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360407040 00000100:00100000:2.0:1788572543.892406:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360407040 00000100:00100000:2.0:1788572543.892476:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4253 00000100:00100000:2.0:1788572543.892487:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35921:x1875453360407040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.892544:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572543.892573:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb162e89 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcbd expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.892598:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:2.0:1788572543.892604:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572543.892637:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.892656:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.892667:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572543.893715:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007fd69d88 reply state 00000000f245f42f transno 4294970480 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.893731:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572543.894778:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35921:x1875453360407040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2287us (2473us total) trans 4294970480 rc 0/0 00000100:00100000:2.0:1788572543.894790:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4253 00000100:00100000:3.0:1788572543.905534:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.905612:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360407168 00000100:00100000:2.0:1788572543.905626:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360407168 00000100:00100000:2.0:1788572543.905690:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1147 00000100:00100000:2.0:1788572543.905701:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360407168:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572543.905981:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572543.905986:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb mtime from disk: 1788572543 -> 1788572517 00002000:00010000:2.0:1788572543.905989:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:2.0:1788572543.905991:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb ctime from disk: 1788572543 -> 1788572526 00002000:00010000:2.0:1788572543.905994:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4d:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572543.906089:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360407168:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 388us (566us total) trans 4294967604 rc 0/0 00000100:00100000:2.0:1788572543.906097:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1147 00010000:00010000:1.0:1788572543.906983:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.906996:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.907053:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.907061:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.907065:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.907074:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.907090:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:1.0:1788572543.907093:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.907098:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.907104:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb162e9e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572543.910678:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967600 is committed 00000001:00080000:3.0:1788572543.910702:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967598 is committed 00000100:00100000:3.0:1788572543.910830:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360406784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 51430us (51667us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.910842:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1145 00000100:00100000:1.0:1788572543.921074:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.921198:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360407424 00000100:00100000:2.0:1788572543.921219:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360407424 00000100:00100000:2.0:1788572543.921286:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4254 00000100:00100000:2.0:1788572543.921298:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35921:x1875453360407424:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.921376:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.921396:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.921407:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.921446:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.921470:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 65 pid: 5837 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.921487:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382362752:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572543.921596:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075480 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.925583:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.925693:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360407680 00000100:00100000:0.0:1788572543.925703:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360407680 00000100:00100000:0.0:1788572543.925771:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4255 00000100:00100000:0.0:1788572543.925783:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35920:x1875453360407680:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.925846:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.925873:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162e5f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc9a expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.925892:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572543.925896:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572543.925925:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.925938:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.925947:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572543.926143:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000031124d36 reply state 000000005c1e7099 transno 4294970481 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.926153:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572543.926346:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35920:x1875453360407680:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 564us (770us total) trans 4294970481 rc 0/0 00000100:00100000:0.0:1788572543.926354:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4255 00000100:00100000:2.0:1788572543.927051:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382362752:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572543.927078:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.937044:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.937136:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572543.937162:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.937178:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.937186:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.937192:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.937197:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.937215:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.937233:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:2.0:1788572543.937235:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572543.937241:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.937247:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162eac lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.937382:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360407808 00000100:00100000:0.0:1788572543.937392:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360407808 00010000:00010000:0.0:1788572543.937424:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 66 pid: 5837 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.937470:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1007 00000100:00100000:0.0:1788572543.937480:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360407808:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.937494:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.937500:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 66 pid: 5837 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.937513:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 66 pid: 5837 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.937530:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb162e35 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc85 expref: 66 pid: 5837 timeout: 991 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.937540:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:0.0:1788572543.937544:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.937554:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.937562:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.937581:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.937626:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360407808:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 148us (590us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.937633:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1007 00010000:00010000:2.0:1788572543.938675:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938688:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938717:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938724:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938731:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938737:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572543.938741:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938748:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938762:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:2.0:1788572543.938764:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572543.938768:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.938775:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162ea5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.938833:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35921:x1875453360407424:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 17537us (17769us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572543.938842:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4254 00000100:00100000:2.0:1788572543.939363:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360407936 00000100:00100000:2.0:1788572543.939371:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360407936 00000100:00100000:2.0:1788572543.939413:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1148 00000100:00100000:2.0:1788572543.939421:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360407936:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572543.939649:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4d:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572543.939653:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4d:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:2.0:1788572543.939656:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4d:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:2.0:1788572543.939659:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4d:0x0] updating lvb ctime from disk: 1788572543 -> 1788572526 00002000:00010000:2.0:1788572543.939661:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4d:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572543.939691:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360407936:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 271us (2555us total) trans 4294967601 rc 0/0 00000100:00100000:2.0:1788572543.939698:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1148 00000100:00100000:0.0:1788572543.945957:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.946000:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.946157:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360408320 00000100:00100000:3.0:1788572543.946170:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360408320 00000100:00100000:3.0:1788572543.946234:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4256 00000100:00100000:3.0:1788572543.946245:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35920:x1875453360408320:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.946320:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.946338:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 64 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.946349:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.946388:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 64 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.946403:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 64 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.946419:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382362880:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.946517:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12900 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.946761:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360408064 00000100:00100000:2.0:1788572543.946766:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360408064 00000100:00100000:2.0:1788572543.946805:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1149 00000100:00100000:2.0:1788572543.946812:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360408064:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572543.946839:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.946856:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.946864:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1788572543.946868:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572543.946873:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 4/1,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.946896:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572543.946908:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.946919:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb649500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382363008:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572543.946964:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c6a7ec00 going to sleep for 11 seconds 00000001:00080000:1.0:1788572543.949401:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967600 is committed 00000001:00080000:1.0:1788572543.949419:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967603 is committed 00000100:00100000:3.0:1788572543.949576:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360406656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 88566us (90353us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.949588:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1146 00000100:00100000:0.0:1788572543.955103:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.955371:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382362880:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572543.955399:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.955564:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb649500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382363008:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572543.955582:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.955654:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360408576 00000100:00100000:1.0:1788572543.955662:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360408576 00000100:00100000:1.0:1788572543.955719:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4257 00000100:00100000:1.0:1788572543.955729:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35922:x1875453360408576:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572543.955785:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572543.955811:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162e43 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc8c expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.955826:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.955831:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.955862:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.955875:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.955884:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572543.956087:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000305a372e reply state 00000000c64ee332 transno 4294970482 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.956095:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572543.956374:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35922:x1875453360408576:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 645us (1281us total) trans 4294970482 rc 0/0 00000100:00100000:1.0:1788572543.956382:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4257 00000100:00100000:2.0:1788572543.960142:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.960234:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572543.960506:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360408704 00000100:00100000:3.0:1788572543.960516:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360408704 00010000:00010000:3.0:1788572543.960560:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 66 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.960597:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360408832 00000100:00100000:3.0:1788572543.960600:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360408832 00010000:00010000:3.0:1788572543.960617:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.960643:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1008 00000100:00100000:3.0:1788572543.960653:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2e52c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360408704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.960667:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.960674:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 66 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960687:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 66 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960708:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb162dda lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc69 expref: 66 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960727:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:3.0:1788572543.960733:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960744:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960752:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960779:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.960823:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2e52c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2347:x1875453360408704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 172us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.960828:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1008 00000100:00100000:3.0:1788572543.960883:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1009 00000100:00100000:3.0:1788572543.960887:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360408832:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.960893:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.960913:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960928:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960941:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d261d483/0x85ace67feb162e97 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efccb expref: 13 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960950:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.960953:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.960957:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960963:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.960977:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.960990:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360408832:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 102us (756us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.960994:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1009 00010000:00010000:3.0:1788572543.961189:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961195:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961226:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961231:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961235:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961239:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572543.961242:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961246:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961252:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:3.0:1788572543.961254:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572543.961257:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961260:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb162eb3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.961294:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35920:x1875453360408320:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 15051us (15293us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572543.961302:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4256 00010000:00010000:3.0:1788572543.961324:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961333:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961353:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961361:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961368:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961374:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572543.961377:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961384:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961398:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.961405:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008ed1c992/0x85ace67feb162eba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.961444:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360408064:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 14632us (15497us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.961462:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1149 00000100:00100000:0.0:1788572543.969178:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.969208:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.969233:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.969261:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360409344 00000100:00100000:0.0:1788572543.969270:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360409344 00000100:00100000:0.0:1788572543.969322:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4258 00000100:00100000:0.0:1788572543.969333:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35921:x1875453360409344:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572543.969435:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969462:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969476:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969714:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969726:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969731:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 4/0,1 mode: --/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969759:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.969772:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.969784:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382363136:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572543.969847:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572543.969981:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360408960 00000100:00100000:2.0:1788572543.969985:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360408960 00000100:00100000:2.0:1788572543.970135:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360409088 00000100:00100000:2.0:1788572543.970140:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360409088 00010000:00010000:1.0:1788572543.970331:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.970341:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.970352:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.970358:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.970362:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.970370:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.970382:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.970385:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.970390:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.970396:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ec1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.970460:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1150 00000100:00100000:3.0:1788572543.970471:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360408960:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572543.970627:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572543.970631:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb mtime from disk: 1788572543 -> 1788572516 00002000:00010000:3.0:1788572543.970633:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:3.0:1788572543.970636:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb ctime from disk: 1788572543 -> 1788572527 00002000:00010000:3.0:1788572543.970638:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4c:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:3.0:1788572543.970674:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360408960:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 205us (1506us total) trans 4294967605 rc 0/0 00000100:00100000:3.0:1788572543.970682:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1150 00000100:00100000:3.0:1788572543.970701:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1151 00000100:00100000:3.0:1788572543.970708:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7663850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360409088:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.970730:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.970751:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.970758:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788572543.970763:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.970767:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.970790:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.970799:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 11 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.970810:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382363264:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.970844:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.975326:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382363136:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572543.975361:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572543.975533:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.975578:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360409472 00000100:00100000:0.0:1788572543.975585:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360409472 00010000:00010000:0.0:1788572543.975758:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.975801:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1010 00000100:00100000:0.0:1788572543.975812:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360409472:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.975828:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.975835:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.975849:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.975874:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162e66 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efca1 expref: 65 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.975891:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:0.0:1788572543.975897:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.975913:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.975921:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.975944:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572543.975978:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.975985:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.976027:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360409472:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 216us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.976035:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1010 00000001:00080000:1.0:1788572543.976184:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970483 00000001:00080000:1.0:1788572543.976329:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970484 00000001:00080000:1.0:1788572543.976380:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970485 00000100:00100000:1.0:1788572543.976475:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382363392:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572543.976521:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572543.976560:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075480 going to sleep for 11 seconds 00000100:00100000:0.0:1788572543.977506:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382363392 00000100:00100000:0.0:1788572543.977515:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382363392 00000100:00100000:0.0:1788572543.977558:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1788572543.977566:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382363392:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572543.977668:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382363392:12345-0@lo:1000:mdt00_004.0 Request processed in 102us (1145us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.977675:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:1.0:1788572543.978302:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967603 gen 2 00000004:00080000:1.0:1788572543.978309:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967595 committed transno 4294967603 00000004:00080000:1.0:1788572543.978329:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786ca80, transno 4294967600 00000004:00080000:1.0:1788572543.978378:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786f800, transno 4294967601 00000004:00080000:1.0:1788572543.979743:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786e300, transno 4294967602 00000004:00080000:1.0:1788572543.979751:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e786dc00, transno 4294967603 00000100:00100000:1.0:1788572543.979762:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382363392:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572543.980795:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.981676:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382363264:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.981697:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572543.982355:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970486 00000100:00100000:1.0:1788572543.982481:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382363520:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572543.982550:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572543.982631:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572543.982715:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360409728 00000100:00100000:3.0:1788572543.982723:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360409728 00000100:00100000:3.0:1788572543.982781:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4259 00000100:00100000:3.0:1788572543.982790:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35922:x1875453360409728:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572543.982863:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.982884:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.982899:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 4/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.982942:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.982957:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.982967:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0d880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382363648:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572543.983252:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572543.983787:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.984076:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382363520 00000100:00100000:2.0:1788572543.984084:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382363520 00000100:00100000:2.0:1788572543.984134:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:2.0:1788572543.984144:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382363520:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572543.984397:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382363520:12345-0@lo:1000:mdt00_004.0 Request processed in 255us (1847us total) trans 4294967606 rc 0/0 00000100:00100000:2.0:1788572543.984405:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:1.0:1788572543.984444:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382363520:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572543.984493:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984503:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984510:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984517:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.984522:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984535:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984553:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7451500) 00010000:00010000:1.0:1788572543.984556:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572543.984562:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 2/0,0 mode: --/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984569:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb162ecf lrc: 0/0,0 mode: --/EX res: [0x200000402:0x8c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984615:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984622:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984628:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984633:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478e400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.984636:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984643:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984651:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627c5716600) 00010000:00010000:1.0:1788572543.984654:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572543.984657:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 2/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984666:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb15f5ef lrc: 0/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984688:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984700:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984709:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984716:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984722:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984729:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984734:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.984737:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984743:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984749:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627c5716600) 00010000:00010000:1.0:1788572543.984752:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572543.984755:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984762:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162ee4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984773:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984779:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984784:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984789:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.984792:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984798:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984806:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627c5716600) 00010000:00010000:1.0:1788572543.984807:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572543.984809:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.984813:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb162ec8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xae:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.984972:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35921:x1875453360409344:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 15640us (15735us total) trans 4294970487 rc 0/0 00000100:00100000:1.0:1788572543.984982:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4258 00000020:00080000:1.0:1788572543.984987:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572543.985585:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360409856 00000100:00100000:3.0:1788572543.985595:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360409856 00010000:00010000:3.0:1788572543.985780:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572543.985827:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1011 00000100:00100000:3.0:1788572543.985837:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360409856:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572543.985863:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572543.985887:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.985902:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.985937:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb162e7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcb6 expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.985952:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.985957:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572543.985964:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.985974:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572543.986037:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572543.986086:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360409856:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 250us (2300us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572543.986094:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1011 00010000:00010000:0.0:1788572543.986341:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986351:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986382:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986390:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986397:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986404:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572543.986407:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986415:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986427:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.986434:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d19e11a3/0x85ace67feb162ed6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.986504:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7663850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360409088:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 15795us (17293us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.986515:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1151 00000100:00100000:2.0:1788572543.995563:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572543.995706:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572543.995866:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360410240 00000100:00100000:0.0:1788572543.995873:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360410240 00000100:00100000:1.0:1788572543.995884:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0d880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382363648:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572543.995911:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.995911:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572543.995965:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1012 00000100:00100000:0.0:1788572543.995976:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360410240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572543.995992:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572543.996000:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.996052:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.996074:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb162d9b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc54 expref: 68 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.996097:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:0.0:1788572543.996103:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.996113:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.996120:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572543.996155:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572543.996222:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360410240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 247us (515us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572543.996229:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1012 00010000:00010000:1.0:1788572543.996478:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996486:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996518:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996525:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996531:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996536:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572543.996541:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996548:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996558:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572543.996561:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572543.996564:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572543.996574:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162edd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572543.996633:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35922:x1875453360409728:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13843us (15843us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572543.996641:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4259 00000100:00100000:3.0:1788572543.997714:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360410112 00000100:00100000:3.0:1788572543.997723:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360410112 00000100:00100000:1.0:1788572543.999878:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.000172:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360410496 00000100:00100000:0.0:1788572544.000180:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360410496 00000100:00100000:0.0:1788572544.000257:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4260 00000100:00100000:0.0:1788572544.000267:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35920:x1875453360410496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.000573:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 3/0,1 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.000593:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.000606:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.003770:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.003801:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 68 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.003811:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.003862:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 68 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.003885:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 68 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.003905:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382363776:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572544.004154:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13170c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.006835:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 911 00000100:00100000:2.0:1788572544.006850:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35921:x1875453360410112:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572544.007817:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x8c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572544.007936:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970489 00000100:00100000:3.0:1788572544.007974:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572544.007987:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.008041:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360410624 00000100:00100000:1.0:1788572544.008050:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360410624 00000100:00100000:3.0:1788572544.008080:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382363776:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.008093:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1152 00000100:00100000:1.0:1788572544.008101:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360410624:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.008106:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.008130:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.008147:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 12 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.008155:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4c:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788572544.008159:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572544.008164:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.008199:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 12 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.008213:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 12 pid: 10416 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.008228:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382363904:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.008308:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00080000:1.0:1788572544.008322:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572544.011968:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572544.012083:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.012104:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.012115:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.012122:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.012129:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.012150:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572544.012268:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35921:x1875453360410112:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 5420us (16716us total) trans 4294970490 rc 0/0 00000100:00100000:2.0:1788572544.012280:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 911 00000020:00080000:2.0:1788572544.012285:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572544.012506:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572544.012513:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.012520:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.012531:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.012540:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.012554:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8c:0x0].0x0 (ffff9627e7a95c00) 00010000:00010000:3.0:1788572544.012558:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572544.012562:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.012568:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb162ef9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.012594:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.012606:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.012608:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572544.012930:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360410752 00000100:00100000:0.0:1788572544.012938:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360410752 00010000:00010000:0.0:1788572544.012970:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 70 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.013043:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1013 00000100:00100000:0.0:1788572544.013052:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360410752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.013067:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.013074:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 70 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.013084:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 70 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.013101:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb162e19 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc7e expref: 70 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.013117:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572544.013122:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.013137:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.013143:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.013164:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.013194:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2350:x1875453360410752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 143us (1239us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.013213:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1013 00010000:00010000:2.0:1788572544.014196:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.014219:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572544.014442:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970488 00000001:00080000:2.0:1788572544.014799:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970491 00000001:00080000:2.0:1788572544.014865:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970492 00000100:00100000:2.0:1788572544.014953:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb64aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382364032:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572544.015047:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.015083:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c6a7ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.015350:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382364032 00000100:00100000:1.0:1788572544.015359:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382364032 00000100:00100000:1.0:1788572544.015418:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:1.0:1788572544.015428:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382364032:12345-0@lo:1000:mdt00_002.0 00000100:00080000:2.0:1788572544.015456:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572544.015519:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382364032:12345-0@lo:1000:mdt00_002.0 Request processed in 92us (471us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.015527:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1788572544.015565:0:5839:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967598 gen 2 00000004:00080000:2.0:1788572544.015571:0:5839:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967597 committed transno 4294967598 00000004:00080000:2.0:1788572544.015589:0:5839:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967598 committed transno 4294967598 00000100:00100000:2.0:1788572544.015601:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb64aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382364032:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1788572544.020444:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.020528:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382363904:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.020556:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.021068:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360410880 00000100:00100000:0.0:1788572544.021076:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360410880 00010000:00010000:0.0:1788572544.021115:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.021147:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1014 00000100:00100000:0.0:1788572544.021156:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360410880:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.021171:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.021539:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.021556:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.021586:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb162e74 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcaf expref: 13 pid: 10416 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.021730:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.021735:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.021741:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.021752:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.021785:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.021842:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360410880:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 685us (1406us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.022144:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1014 00010000:00010000:3.0:1788572544.022221:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022232:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022260:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022273:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022281:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022464:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.022469:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022488:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022501:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.022508:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb162f00 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.022575:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360410624:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 14474us (14602us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.022584:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1152 00000001:00080000:2.0:1788572544.023167:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970493 00000100:00100000:2.0:1788572544.025228:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb64aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382364160:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572544.026170:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.026255:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b79a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.028445:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382364160 00000100:00100000:0.0:1788572544.028467:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382364160 00000100:00100000:0.0:1788572544.028585:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:0.0:1788572544.028596:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382364160:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1788572544.028981:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.029125:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360411136 00000100:00100000:2.0:1788572544.029130:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360411136 00000100:00100000:2.0:1788572544.029172:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4261 00000100:00100000:2.0:1788572544.029181:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35922:x1875453360411136:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.029260:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029280:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029304:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029540:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029551:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029566:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 4/0,1 mode: --/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029613:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.029626:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.029856:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382364288:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.029947:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0776c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.030865:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382364160:12345-0@lo:1000:mdt00_002.0 Request processed in 2267us (4692us total) trans 4294967602 rc 0/0 00000100:00100000:1.0:1788572544.030875:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1788572544.030949:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb64aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382364160:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1788572544.030986:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.030994:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031057:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031065:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.031070:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031079:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031097:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627e7451700) 00010000:00010000:2.0:1788572544.031100:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572544.031105:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031111:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162ef2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x8b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031144:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031150:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031155:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031160:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.031163:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031169:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031176:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:2.0:1788572544.031179:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572544.031182:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031187:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb15f539 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031531:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031543:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031552:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031560:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031566:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031572:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031577:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.031579:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031585:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031592:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:2.0:1788572544.031594:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.031597:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031603:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f15 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031614:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031620:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031626:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031631:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.031633:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031639:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031646:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:2.0:1788572544.031649:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572544.031651:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.031657:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162eeb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.031823:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35920:x1875453360410496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 31555us (31954us total) trans 4294970494 rc 0/0 00000100:00100000:2.0:1788572544.031834:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4260 00000020:00080000:2.0:1788572544.031837:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572544.035831:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.035968:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382364288:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572544.035996:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.037482:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.038290:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360411264 00000100:00100000:0.0:1788572544.038301:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360411264 00010000:00010000:1.0:1788572544.038426:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.038477:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1015 00000100:00100000:1.0:1788572544.038490:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec087050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2350:x1875453360411264:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572544.038505:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.038512:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038525:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038547:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb162dbe lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efc62 expref: 74 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038567:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:1.0:1788572544.038572:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038581:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038589:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038619:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572544.038684:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec087050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2350:x1875453360411264:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 195us (2862us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.038691:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1015 00010000:00010000:1.0:1788572544.038753:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.038760:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572544.039152:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970495 00000001:00080000:1.0:1788572544.039464:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970496 00000001:00080000:1.0:1788572544.039522:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970497 00000100:00100000:1.0:1788572544.039615:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382364416:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572544.039670:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572544.039756:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5e00c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.041210:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360411520 00000100:00100000:3.0:1788572544.041221:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360411520 00000100:00100000:3.0:1788572544.041278:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 912 00000100:00100000:3.0:1788572544.041287:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35920:x1875453360411520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572544.041614:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x8b:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572544.041657:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970499 00010000:00010000:3.0:1788572544.041701:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.041717:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.041729:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.041737:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.041743:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.041749:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.041762:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572544.041851:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd3450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35920:x1875453360411520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 563us (4372us total) trans 4294970500 rc 0/0 00000100:00100000:3.0:1788572544.041859:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 912 00000020:00080000:3.0:1788572544.041864:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572544.042687:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572544.042695:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.042703:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.042707:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.042716:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.042730:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8b:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:0.0:1788572544.042733:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572544.042738:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.042744:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f1c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.042761:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.042763:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.042765:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572544.049312:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382364416 00000100:00100000:2.0:1788572544.049329:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382364416 00000100:00100000:3.0:1788572544.051860:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:3.0:1788572544.051875:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382364416:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572544.052195:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382364416:12345-0@lo:1000:mdt00_000.0 Request processed in 323us (12524us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.052314:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382364416:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1788572544.052777:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970498 00000100:00100000:0.0:1788572544.052869:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ea300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382364544:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572544.052913:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572544.052950:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.053170:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:3.0:1788572544.053192:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382364544 00000100:00100000:3.0:1788572544.053197:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382364544 00000100:00100000:3.0:1788572544.053232:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1788572544.053239:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382364544:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572544.053538:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5837:x1875453382364544:12345-0@lo:1000:mdt00_000.0 Request processed in 298us (624us total) trans 4294967607 rc 0/0 00000100:00100000:3.0:1788572544.053546:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1788572544.053699:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ea300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382364544:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1788572544.053735:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053752:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053760:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053766:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.053772:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053781:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053797:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc16000) 00010000:00010000:0.0:1788572544.053800:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.053806:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053812:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb162f0e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x8a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053846:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053852:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053857:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053861:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.053864:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053870:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053876:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:0.0:1788572544.053878:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.053882:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053891:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb15f658 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.053912:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054146:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054158:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054167:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054172:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054178:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054183:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.054186:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054192:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054498:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:0.0:1788572544.054502:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.054506:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054513:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f23 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054525:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054532:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054537:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054542:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.054545:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054551:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054559:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:0.0:1788572544.054561:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572544.054564:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.054569:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb162f07 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.054731:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35922:x1875453360411136:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 25548us (25748us total) trans 4294970501 rc 0/0 00000100:00100000:0.0:1788572544.054741:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4261 00000020:00080000:0.0:1788572544.054746:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1788572544.054898:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572544.054903:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572544.054965:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 145 count 101 00000040:00080000:1.0:1788572544.054971:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:1.0:1788572544.055110:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f1f80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382364672:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572544.055154:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87fb40 going to sleep for 11 seconds 00000100:00080000:2.0:1788572544.056168:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572544.058897:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f1f80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382364672:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572544.063721:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.063824:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.063932:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.064050:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360411648 00000100:00100000:2.0:1788572544.064060:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360411648 00000100:00100000:2.0:1788572544.064121:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360412032 00000100:00100000:2.0:1788572544.064125:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360412032 00000100:00100000:2.0:1788572544.064154:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4262 00000100:00100000:2.0:1788572544.064163:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35923:x1875453360411648:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572544.064196:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb0:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.064259:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4263 00000100:00100000:0.0:1788572544.064268:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35924:x1875453360412032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.064300:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb6:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572544.064312:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35923:x1875453360411648:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 150us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.064320:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4262 00000100:00100000:1.0:1788572544.064412:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360412160 00000100:00100000:1.0:1788572544.064416:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360412160 00000100:00100000:1.0:1788572544.064444:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4264 00000100:00100000:1.0:1788572544.064470:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35925:x1875453360412160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572544.064485:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb3:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572544.064533:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35925:x1875453360412160:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 64us (611us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.064540:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4264 00000100:00100000:0.0:1788572544.064631:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35924:x1875453360412032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 364us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.064748:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4263 00000100:00100000:0.0:1788572544.068427:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.068514:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360412288 00000100:00100000:2.0:1788572544.068524:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360412288 00000100:00100000:2.0:1788572544.068584:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4265 00000100:00100000:2.0:1788572544.068596:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35925:x1875453360412288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.068679:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068697:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068709:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068724:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068737:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068744:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068772:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068780:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068787:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.068828:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb3:0x0]: returning size 3 00000004:00200000:2.0:1788572544.068832:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.068856:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068869:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068876:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068883:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068889:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.068893:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068901:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068914:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.068917:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.068922:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068928:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb162f2a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068942:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068947:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068952:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068957:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.068960:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068965:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068972:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x4b1d977c (ffff9627e7a95e00) 00010000:00010000:2.0:1788572544.068974:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.068977:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.068987:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb162f31 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4b1d977c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069047:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069055:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069061:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069066:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.069068:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069076:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069084:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:2.0:1788572544.069087:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.069089:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.069095:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb162f38 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.069215:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35925:x1875453360412288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 620us (794us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.069223:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4265 00000100:00100000:1.0:1788572544.070555:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.070670:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.070706:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.070869:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360412544 00000100:00100000:0.0:1788572544.070878:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360412544 00000100:00100000:0.0:1788572544.070928:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360412416 00000100:00100000:0.0:1788572544.070932:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360412416 00000100:00100000:0.0:1788572544.070955:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4266 00000100:00100000:0.0:1788572544.070964:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35923:x1875453360412544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.071058:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071071:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071079:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071091:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071099:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071103:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071124:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071129:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071132:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.071164:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb0:0x0]: returning size 3 00000004:00200000:0.0:1788572544.071168:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572544.071180:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071188:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071192:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071197:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071206:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.071209:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071215:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071222:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572544.071224:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.071227:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071230:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb162f3f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071238:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071242:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071245:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071248:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.071250:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071253:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071256:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xb4d28fa5 (ffff9627e7451000) 00010000:00010000:0.0:1788572544.071257:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.071259:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071265:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162f46 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb4d28fa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071276:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071279:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071282:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071293:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.071295:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071299:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071303:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627c5716400) 00010000:00010000:0.0:1788572544.071304:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.071306:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.071310:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb162f4d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.071557:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4267 00000100:00100000:3.0:1788572544.071569:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35924:x1875453360412416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.071602:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071612:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071620:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071632:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071639:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071645:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071663:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071671:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071677:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.071697:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb6:0x0]: returning size 3 00000004:00200000:3.0:1788572544.071700:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.071707:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071716:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071722:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071728:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071733:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.071735:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071741:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071749:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572544.071751:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.071753:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071764:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb162f54 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071774:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071779:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071785:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071790:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.071794:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071799:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071806:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xf8c9e8a8 (ffff9627ffc51300) 00010000:00010000:3.0:1788572544.071809:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.071812:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071817:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb162f5b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xf8c9e8a8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071829:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071835:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071841:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071860:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.071863:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071870:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071877:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572544.071879:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.071882:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.071887:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb162f62 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.071921:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35924:x1875453360412416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 356us (1214us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.071929:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4267 00000100:00100000:0.0:1788572544.072001:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35923:x1875453360412544:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 504us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.072091:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4266 00000100:00100000:3.0:1788572544.072366:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.072389:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360412800 00000100:00100000:3.0:1788572544.072395:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360412800 00000100:00100000:3.0:1788572544.072429:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4268 00000100:00100000:3.0:1788572544.072437:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35925:x1875453360412800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.072458:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.072470:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162f69 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcd2 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072571:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072583:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072591:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.072600:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb3:0x0]: returning size 3 00010000:00010000:3.0:1788572544.072694:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072701:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072707:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072712:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.072715:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072721:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072730:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:3.0:1788572544.072732:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.072736:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072747:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162f70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072781:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162f69 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcd2 expref: 80 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072791:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572544.072794:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572544.072797:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162f69 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcd2 expref: 80 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.072806:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572544.072833:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35925:x1875453360412800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 397us (469us total) trans 4294970502 rc 301/301 00000100:00100000:3.0:1788572544.072840:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4268 00000100:00100000:2.0:1788572544.073088:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360411904 00000100:00100000:2.0:1788572544.073094:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360411904 00000100:00100000:3.0:1788572544.073231:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 913 00000100:00100000:3.0:1788572544.073240:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35922:x1875453360411904:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572544.073551:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x8a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572544.073591:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970503 00010000:00010000:3.0:1788572544.073628:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073639:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073647:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073659:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073665:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073670:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073686:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788572544.073718:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572544.073724:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073729:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.073733:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073740:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073749:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8a:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.073752:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572544.073755:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 1/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073760:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb162f7e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.073772:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.073774:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.073776:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572544.073812:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35922:x1875453360411904:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 574us (3270us total) trans 4294970504 rc 0/0 00000100:00100000:3.0:1788572544.073819:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 913 00000020:00080000:3.0:1788572544.073823:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788572544.076425:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.076481:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.076552:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360413056 00000100:00100000:3.0:1788572544.076563:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360413056 00000100:00100000:3.0:1788572544.076617:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360413312 00000100:00100000:3.0:1788572544.076620:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360413312 00000100:00100000:3.0:1788572544.076651:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4269 00000100:00100000:3.0:1788572544.076660:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35923:x1875453360413056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.076682:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.076702:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162f85 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcd9 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076802:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076817:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076828:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.076840:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb0:0x0]: returning size 3 00010000:00010000:3.0:1788572544.076942:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076950:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076960:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076966:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.076970:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076978:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.076990:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627c5716400) 00010000:00010000:3.0:1788572544.076993:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.076996:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077028:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162f8c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077072:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162f85 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcd9 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077083:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572544.077086:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572544.077090:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162f85 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcd9 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077100:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cce00, rc 0) 00000100:00100000:3.0:1788572544.077152:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35923:x1875453360413056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 492us (734us total) trans 4294970505 rc 301/301 00000100:00100000:3.0:1788572544.077163:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4269 00000100:00100000:3.0:1788572544.077181:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4270 00000100:00100000:3.0:1788572544.077188:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35924:x1875453360413312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.077196:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.077204:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162f9a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efce0 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077245:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077254:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077261:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.077268:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb6:0x0]: returning size 3 00010000:00010000:3.0:1788572544.077309:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077316:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077321:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077326:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.077329:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077338:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077346:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572544.077348:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.077351:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077356:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb162fa1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077377:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162f9a lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efce0 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077383:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572544.077385:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572544.077388:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb162f9a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efce0 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.077396:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572544.077416:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35924:x1875453360413312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 229us (935us total) trans 4294970506 rc 301/301 00000100:00100000:3.0:1788572544.077421:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4270 00000100:00100000:0.0:1788572544.081500:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.081583:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360413440 00000100:00100000:3.0:1788572544.081594:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360413440 00000100:00100000:3.0:1788572544.081662:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4271 00000100:00100000:3.0:1788572544.081674:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35925:x1875453360413440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572544.081713:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb3:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572544.081835:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35925:x1875453360413440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 162us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.081842:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4271 00000100:00100000:1.0:1788572544.081982:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.082407:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360413568 00000100:00100000:0.0:1788572544.082411:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360413568 00000100:00100000:0.0:1788572544.082440:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4272 00000100:00100000:0.0:1788572544.082446:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35923:x1875453360413568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.082502:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb0:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.082576:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35923:x1875453360413568:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 129us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.082583:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4272 00000100:00100000:1.0:1788572544.083895:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.084086:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360413696 00000100:00100000:0.0:1788572544.084093:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360413696 00000100:00100000:0.0:1788572544.084138:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4273 00000100:00100000:0.0:1788572544.084148:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35924:x1875453360413696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.084176:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb6:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.084396:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35924:x1875453360413696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 248us (506us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.084403:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4273 00000100:00100000:2.0:1788572544.087528:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.087626:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360413824 00000100:00100000:1.0:1788572544.087702:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360413824 00000100:00100000:1.0:1788572544.087762:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4274 00000100:00100000:1.0:1788572544.087772:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35925:x1875453360413824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.087796:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.087817:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162faf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efce7 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.087886:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.087901:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.087912:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.087934:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.087942:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.087948:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.088995:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb162fc4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089072:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb162fc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089078:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb162fc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572544.089105:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:1.0:1788572544.089107:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572544.089122:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb162fc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089131:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089136:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089141:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089145:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.089148:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089152:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089161:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572544.089163:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.089166:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089173:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb162fb6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089182:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089185:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089189:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089192:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.089193:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089196:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089209:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627c5716600) 00010000:00010000:1.0:1788572544.089212:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.089215:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089219:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb162fbd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089252:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb162faf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efce7 expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089270:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb162fc4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efce7 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.089276:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.089277:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572544.089279:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572544.089441:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35925:x1875453360413824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1670us (1923us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.089456:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4274 00000100:00100000:3.0:1788572544.094621:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.094713:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360414080 00000100:00100000:1.0:1788572544.094724:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360414080 00000100:00100000:1.0:1788572544.094792:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4275 00000100:00100000:1.0:1788572544.094803:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35923:x1875453360414080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.094825:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.094827:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572544.094855:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb162fc4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efce7 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.094873:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.094879:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.094894:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162fcb lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcee expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572544.095385:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4e:0x0], index 0: 0 00010000:00010000:1.0:1788572544.095545:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162fd2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.095557:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162fd2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.095564:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162fd2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.095603:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb162fcb lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcee expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.095613:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162fd2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcee expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.095618:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:1.0:1788572544.095620:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572544.095622:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96a00, rc 0) 00000100:00100000:1.0:1788572544.095676:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35923:x1875453360414080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 876us (1065us total) trans 4294970507 rc 0/0 00000100:00100000:1.0:1788572544.095684:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4275 00000100:00100000:2.0:1788572544.097079:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.097117:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360414208 00000100:00100000:3.0:1788572544.097125:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360414208 00000100:00100000:3.0:1788572544.097167:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4276 00000100:00100000:3.0:1788572544.097177:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35924:x1875453360414208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.097192:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.097505:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162fe0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcf5 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097590:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097603:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097614:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097629:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097635:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097642:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097912:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162ff5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097936:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162ff5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.097944:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162ff5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.097971:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:3.0:1788572544.097975:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.097991:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162ff5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098031:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098039:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098046:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098052:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.098057:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098064:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098076:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572544.098080:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.098085:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098091:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb162fe7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098103:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098109:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098114:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098119:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.098122:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098128:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098135:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.098138:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.098140:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098147:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb162fee lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098178:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb162fe0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcf5 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098191:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162ff5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcf5 expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.098208:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572544.098211:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.098213:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572544.098276:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35924:x1875453360414208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1101us (1200us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.098283:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4276 00000100:00100000:3.0:1788572544.101342:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.101462:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360414464 00000100:00100000:0.0:1788572544.101471:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360414464 00000100:00100000:0.0:1788572544.101520:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4277 00000100:00100000:0.0:1788572544.101528:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35925:x1875453360414464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.101551:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.101569:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162ffc lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efcfc expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572544.102171:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4e:0x0], index 1: 0 00010000:00010000:0.0:1788572544.102482:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb163003 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.102495:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb163003 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.102504:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb163003 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.102549:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb162ffc lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcfc expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.102562:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb163003 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcfc expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.102569:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627e7451000) 00010000:00010000:0.0:1788572544.102571:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572544.102573:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:0.0:1788572544.102711:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35925:x1875453360414464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1183us (1424us total) trans 4294970508 rc 0/0 00000100:00100000:0.0:1788572544.102717:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4277 00000100:00100000:0.0:1788572544.105459:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.105516:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360414720 00000100:00100000:1.0:1788572544.105524:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360414720 00000100:00100000:1.0:1788572544.105575:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4278 00000100:00100000:1.0:1788572544.105582:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35923:x1875453360414720:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.105632:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572544.105661:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb162fd2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcee expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105675:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:1.0:1788572544.105685:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb162ff5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcf5 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105691:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.105694:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.105713:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105723:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105729:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105738:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105742:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105746:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105935:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105943:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105949:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105955:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.105961:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105969:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105978:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.105980:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.105982:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105987:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163011 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105993:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.105997:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.106000:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.106159:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.106161:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.106165:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.106172:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x92803260 (ffff9627c5716600) 00010000:00010000:1.0:1788572544.106173:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.106176:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.106179:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163018 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x92803260 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.106284:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35923:x1875453360414720:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 699us (838us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572544.106292:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4278 00000100:00100000:2.0:1788572544.108293:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.109782:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.109816:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360414848 00000100:00100000:0.0:1788572544.109826:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360414848 00000100:00100000:3.0:1788572544.109845:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360415104 00000100:00100000:3.0:1788572544.109849:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360415104 00000100:00100000:3.0:1788572544.109920:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4279 00000100:00100000:3.0:1788572544.109931:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35924:x1875453360415104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.109951:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.109972:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb16301f lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd0a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572544.110441:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4f:0x0], index 0: 0 00010000:00010000:3.0:1788572544.110645:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb163026 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110660:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb163026 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110672:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb163026 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110737:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb16301f lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd0a expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110757:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb163026 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd0a expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110766:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627ffc51300) 00010000:00010000:3.0:1788572544.110769:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572544.110773:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:3.0:1788572544.110838:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35924:x1875453360415104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 908us (1062us total) trans 4294970509 rc 0/0 00000100:00100000:3.0:1788572544.110845:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4279 00000100:00100000:3.0:1788572544.110872:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4280 00000100:00100000:3.0:1788572544.110881:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35923:x1875453360414848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.110892:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.110904:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163034 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd03 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110925:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110934:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.110941:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.111049:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163034 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.111062:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.111069:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:3.0:1788572544.111072:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.111074:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc000, rc 0) 00000100:00100000:3.0:1788572544.111110:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35923:x1875453360414848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 230us (2825us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.111116:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4280 00000100:00100000:1.0:1788572544.113661:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.113736:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360415360 00000100:00100000:0.0:1788572544.113746:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360415360 00000100:00100000:0.0:1788572544.113797:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4281 00000100:00100000:0.0:1788572544.113804:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35923:x1875453360415360:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.113914:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.113929:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.113938:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572544.114361:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].66, 88 off13520 00000004:00080000:0.0:1788572544.114370:0:5838:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 892183364887ns, next at 893183363212ns, rc = 0 00000004:00010000:0.0:1788572544.114440:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d4325374 reply state 000000001d63b893 transno 4294970510 ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.114459:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572544.115245:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35923:x1875453360415360:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1436us (1590us total) trans 4294970510 rc 0/0 00000100:00100000:0.0:1788572544.115255:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4281 00000100:00100000:2.0:1788572544.115899:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.115940:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.116001:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360415616 00000100:00100000:3.0:1788572544.116044:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360415616 00000100:00100000:3.0:1788572544.116077:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360415744 00000100:00100000:3.0:1788572544.116079:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360415744 00000100:00100000:3.0:1788572544.116094:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4282 00000100:00100000:3.0:1788572544.116100:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35924:x1875453360415616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.116114:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.116128:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163049 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd11 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116175:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116186:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116196:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116206:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116211:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116215:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116480:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116490:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116497:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.116523:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:3.0:1788572544.116527:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.116546:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116557:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116564:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116575:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116583:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.116588:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116596:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116609:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572544.116612:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.116618:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116625:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163050 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116648:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116655:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116661:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116666:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.116668:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116678:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116687:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.116689:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.116692:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116697:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163057 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116729:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163049 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 94 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116743:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 93 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116751:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572544.116753:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.116756:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cca00, rc 0) 00000100:00100000:3.0:1788572544.116859:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35924:x1875453360415616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 755us (961us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.116866:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4282 00000100:00100000:3.0:1788572544.116887:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4283 00000100:00100000:3.0:1788572544.116895:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35925:x1875453360415744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.116927:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.116942:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb163003 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efcfc expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116952:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627e7451000) 00010000:00010000:3.0:1788572544.116960:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572544.116970:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116980:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.116988:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.117040:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.117054:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 91 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.117068:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382364800:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.117124:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00010000:00010000:0.0:1788572544.118657:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.118669:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.118676:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.118681:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.118685:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.118693:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.118706:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:0.0:1788572544.118708:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572544.118713:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.118719:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163042 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.118798:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.118930:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360415872 00000100:00100000:2.0:1788572544.118937:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360415872 00000100:00100000:2.0:1788572544.118981:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4284 00000100:00100000:2.0:1788572544.118991:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35923:x1875453360415872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.119143:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.119157:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd18 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.119199:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.119207:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.119219:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:2.0:1788572544.119221:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572544.119223:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04c00, rc 0) 00000100:00100000:2.0:1788572544.119264:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35923:x1875453360415872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 274us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.119270:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4284 00000100:00100000:2.0:1788572544.122553:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.122834:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382364800:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.122838:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572544.122922:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.123083:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572544.123293:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360416128 00000100:00100000:1.0:1788572544.123301:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360416128 00010000:00010000:1.0:1788572544.123338:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 92 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.123385:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1016 00000100:00100000:1.0:1788572544.123394:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:2349:x1875453360416128:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572544.123410:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.123417:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 92 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.123430:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 92 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.123462:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16305e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd11 expref: 92 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.123476:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.123480:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.123492:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.123500:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.123530:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572544.123579:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:2349:x1875453360416128:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 186us (740us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.123587:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1016 00010000:00010000:2.0:1788572544.124195:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124211:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124230:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124237:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124242:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124433:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124438:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124443:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124446:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.124462:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124470:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124480:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572544.124483:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.124487:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124494:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163065 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124505:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124510:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124516:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124521:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.124524:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124529:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124537:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x26fa6c92 (ffff9627e7a95e00) 00010000:00010000:2.0:1788572544.124540:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.124547:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.124553:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163073 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x26fa6c92 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.124627:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35925:x1875453360415744:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 7731us (8685us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572544.124982:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360416000 00000100:00100000:3.0:1788572544.124990:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360416000 00000100:00080000:1.0:1788572544.125081:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572544.125142:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1153 00000100:00100000:3.0:1788572544.125152:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360416000:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.125412:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360416000:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 261us (2866us total) trans 4294967603 rc 0/0 00000100:00100000:3.0:1788572544.125419:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1153 00000100:00100000:2.0:1788572544.125483:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4283 00000100:00100000:1.0:1788572544.127485:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.127574:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360416256 00000100:00100000:3.0:1788572544.127584:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360416256 00000100:00100000:3.0:1788572544.127687:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4285 00000100:00100000:3.0:1788572544.127698:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35925:x1875453360416256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.127718:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.127741:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb16307a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd1f expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.127780:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.127791:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.127799:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.127898:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb16307a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.127914:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.127923:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.127937:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.127941:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cdc00, rc 0) 00000100:00100000:3.0:1788572544.127997:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35925:x1875453360416256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 303us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.128036:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4285 00000100:00100000:2.0:1788572544.129391:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.129426:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.129432:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360416512 00000100:00100000:1.0:1788572544.129440:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360416512 00000100:00100000:1.0:1788572544.129549:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4286 00000100:00100000:1.0:1788572544.129560:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35924:x1875453360416512:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.129561:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360416768 00000100:00100000:0.0:1788572544.129565:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360416768 00000100:00100000:0.0:1788572544.129593:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 914 00000100:00100000:0.0:1788572544.129600:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35923:x1875453360416768:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.129606:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.129626:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb163026 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd0a expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129640:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627ffc51300) 00010000:00010000:1.0:1788572544.129649:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.129669:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129680:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129689:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129704:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129711:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129717:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129931:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129940:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129946:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129951:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.129955:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129963:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129972:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.129975:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.129979:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129985:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163088 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.129995:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.130000:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.130028:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.130032:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.130035:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.130040:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.130048:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x990d0d07 (ffff9627c5716600) 00010000:00010000:1.0:1788572544.130051:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.130054:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.130059:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16308f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x990d0d07 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.130127:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35924:x1875453360416512:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 567us (738us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572544.130135:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4286 00000100:00100000:2.0:1788572544.131664:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.131680:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35923:x1875453360416768:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 2079us (2251us total) trans 4294970511 rc 0/0 00000100:00100000:0.0:1788572544.131690:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 914 00000100:00100000:2.0:1788572544.131755:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360417024 00000100:00100000:2.0:1788572544.131763:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360417024 00000100:00100000:2.0:1788572544.131813:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4287 00000100:00100000:2.0:1788572544.131821:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35925:x1875453360417024:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.131912:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.131928:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.131938:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572544.132263:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].64, 88 off13416 00000004:00010000:2.0:1788572544.132343:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000368ad0e2 reply state 0000000002e39a1a transno 4294970512 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.132352:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572544.132414:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35925:x1875453360417024:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 592us (756us total) trans 4294970512 rc 0/0 00000100:00100000:2.0:1788572544.132421:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4287 00000100:00100000:2.0:1788572544.134283:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.134384:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360417152 00000100:00100000:1.0:1788572544.134393:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360417152 00000100:00100000:1.0:1788572544.134458:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4288 00000100:00100000:1.0:1788572544.134471:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35924:x1875453360417152:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.134489:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.134507:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16309d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd26 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.134534:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.134545:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.134552:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.134631:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16309d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.134654:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 94 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.134660:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:1.0:1788572544.134662:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572544.134664:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:1.0:1788572544.134747:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35924:x1875453360417152:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 279us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.134752:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4288 00000100:00100000:1.0:1788572544.137459:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.137752:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360417408 00000100:00100000:2.0:1788572544.137763:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360417408 00000100:00100000:2.0:1788572544.137830:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4289 00000100:00100000:2.0:1788572544.137840:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35923:x1875453360417408:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.137967:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35923:x1875453360417408:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 127us (527us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.137974:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4289 00010000:00010000:0.0:1788572544.138878:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.138891:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.138902:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.138907:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.138912:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.138931:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.138949:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:0.0:1788572544.138952:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572544.138957:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.138964:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163096 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.140081:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.140372:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360417536 00000100:00100000:3.0:1788572544.140382:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360417536 00000100:00100000:3.0:1788572544.140436:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4290 00000100:00100000:3.0:1788572544.140446:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35925:x1875453360417536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.140486:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.140506:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd2d expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.140580:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.140593:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.140613:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.140616:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.140618:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572544.141790:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35925:x1875453360417536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1339us (1706us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.141801:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4290 00000100:00100000:0.0:1788572544.146404:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.147169:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360417664 00000100:00100000:2.0:1788572544.147180:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360417664 00000100:00100000:2.0:1788572544.147342:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4291 00000100:00100000:2.0:1788572544.147354:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35923:x1875453360417664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.147378:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.147400:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1630b2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd34 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.147484:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1630b9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.147500:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1630b9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.147511:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1630b9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.147549:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.147569:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1630b9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.147596:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1630b2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd34 expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.147615:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1630b9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd34 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.147630:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:2.0:1788572544.147633:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572544.147708:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:2.0:1788572544.147763:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35923:x1875453360417664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 412us (1369us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.147771:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4291 00000100:00100000:0.0:1788572544.151632:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.151781:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360417920 00000100:00100000:2.0:1788572544.151796:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360417920 00000100:00100000:2.0:1788572544.151863:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4292 00000100:00100000:2.0:1788572544.151874:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35924:x1875453360417920:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.151987:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.152032:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.152046:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572544.152444:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].67, 88 off13608 00000004:00010000:2.0:1788572544.152550:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eec02a05 reply state 00000000e81abd3a transno 4294970513 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.152560:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572544.152851:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35924:x1875453360417920:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 976us (1232us total) trans 4294970513 rc 0/0 00000100:00100000:2.0:1788572544.152860:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4292 00000100:00100000:0.0:1788572544.160293:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.160418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572544.160507:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.160520:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.160529:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.160535:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.160541:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.160549:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.160566:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:2.0:1788572544.160569:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572544.160574:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.160581:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1630c0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.160778:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360418176 00000100:00100000:3.0:1788572544.160788:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360418176 00000100:00100000:3.0:1788572544.160847:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4293 00000100:00100000:3.0:1788572544.160857:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35924:x1875453360418176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.160879:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.160898:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd3b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.160972:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 97 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.160986:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 97 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.161151:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:3.0:1788572544.161154:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.161157:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd200, rc 0) 00000100:00100000:3.0:1788572544.161223:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35924:x1875453360418176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 369us (806us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.161229:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4293 00000100:00100000:1.0:1788572544.161583:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360418048 00000100:00100000:1.0:1788572544.161589:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360418048 00000100:00100000:1.0:1788572544.161630:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1154 00000100:00100000:1.0:1788572544.162216:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360418048:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.162563:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360418048:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 924us (2278us total) trans 4294967608 rc 0/0 00000100:00100000:0.0:1788572544.162572:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1154 00000100:00100000:0.0:1788572544.163418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.163497:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360418304 00000100:00100000:3.0:1788572544.163504:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360418304 00000100:00100000:3.0:1788572544.163550:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1155 00000100:00100000:3.0:1788572544.163559:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360418304:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.163573:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.163589:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd42 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572544.163613:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737f, blocks: 0x1 00010000:00010000:3.0:1788572544.163627:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=892s v=(0 0) 00010000:00010000:3.0:1788572544.163632:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.163710:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.163736:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.163749:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cce00, rc 0) 00000100:00100000:3.0:1788572544.163863:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360418304:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 303us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.163870:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1155 00000100:00100000:3.0:1788572544.169262:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.169352:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360418432 00000100:00100000:0.0:1788572544.169363:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360418432 00000100:00100000:0.0:1788572544.169422:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1156 00000100:00100000:0.0:1788572544.169432:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360418432:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.169769:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360418432:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 337us (515us total) trans 4294967604 rc 0/0 00000100:00100000:0.0:1788572544.169777:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1156 00000100:00100000:0.0:1788572544.171741:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.171895:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360418688 00000100:00100000:3.0:1788572544.171905:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360418688 00000100:00100000:3.0:1788572544.171963:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 915 00000100:00100000:3.0:1788572544.171973:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35925:x1875453360418688:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.172434:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35925:x1875453360418688:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 461us (703us total) trans 4294970514 rc 0/0 00000100:00100000:3.0:1788572544.172442:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 915 00000100:00100000:1.0:1788572544.174071:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.174150:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360418944 00000100:00100000:2.0:1788572544.174159:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360418944 00000100:00100000:2.0:1788572544.174220:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4294 00000100:00100000:2.0:1788572544.174229:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35923:x1875453360418944:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.174317:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35923:x1875453360418944:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 88us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.174325:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4294 00000100:00100000:2.0:1788572544.177258:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.177297:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.177410:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360419328 00000100:00100000:0.0:1788572544.177419:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360419328 00000100:00100000:0.0:1788572544.177481:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 916 00000100:00100000:0.0:1788572544.177490:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35924:x1875453360419328:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.177945:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35924:x1875453360419328:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 453us (697us total) trans 4294970515 rc 0/0 00000100:00100000:0.0:1788572544.177953:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 916 00000100:00100000:3.0:1788572544.179627:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360419200 00000100:00100000:3.0:1788572544.179651:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360419200 00000100:00100000:3.0:1788572544.179720:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1157 00000100:00100000:3.0:1788572544.179730:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35923:x1875453360419200:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.179747:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.179767:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd49 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.179796:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.179806:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.179810:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.179815:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.179845:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.179859:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 14 pid: 6681 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.179873:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382364928:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.179954:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.180313:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.180344:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.180365:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360419584 00000100:00100000:3.0:1788572544.180369:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360419584 00000100:00100000:3.0:1788572544.180395:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360419840 00000100:00100000:3.0:1788572544.180397:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360419840 00000100:00100000:3.0:1788572544.180413:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4295 00000100:00100000:3.0:1788572544.180419:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35925:x1875453360419584:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.180491:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35925:x1875453360419584:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 72us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.180498:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4295 00000100:00100000:3.0:1788572544.180512:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4296 00000100:00100000:3.0:1788572544.180518:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35924:x1875453360419840:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.180542:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35924:x1875453360419840:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 25us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.180548:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4296 00000100:00100000:1.0:1788572544.183977:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.184132:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.184155:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.184177:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420096 00000100:00100000:1.0:1788572544.184184:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420096 00000100:00100000:1.0:1788572544.184221:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420224 00000100:00100000:1.0:1788572544.184223:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420224 00000100:00100000:1.0:1788572544.184235:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4297 00000100:00100000:1.0:1788572544.184242:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35925:x1875453360420096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.184260:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.184278:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1630dc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd50 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184348:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1630e3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184363:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1630e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184374:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1630e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572544.184410:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572544.184425:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1630e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184460:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1630dc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd50 expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184481:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1630e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd50 expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184494:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:1.0:1788572544.184497:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572544.184500:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96000, rc 0) 00000100:00100000:0.0:1788572544.184527:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360419968 00000100:00100000:0.0:1788572544.184531:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360419968 00010000:00010000:0.0:1788572544.184553:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 15 pid: 6681 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.184554:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35925:x1875453360420096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 312us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.184561:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4297 00000100:00100000:0.0:1788572544.184580:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1017 00000100:00100000:1.0:1788572544.184580:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4298 00000100:00100000:0.0:1788572544.184587:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360419968:12345-192.168.202.40@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1788572544.184587:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35924:x1875453360420224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.184594:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.184597:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.184602:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1630ea lrc: 2/0,0 mode: --/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd57 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.184616:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 15 pid: 6681 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.184629:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 15 pid: 6681 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184631:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1630f1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.184651:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.184655:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.184663:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.184683:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.184724:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.184739:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb473800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382365056:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572544.184765:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1630f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184773:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1630f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.184796:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1317c00 going to sleep for 11 seconds 00000004:00200000:1.0:1788572544.184796:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572544.184806:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1630f1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184823:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1630ea lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd57 expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184835:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1630f1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd57 expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.184842:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:1.0:1788572544.184845:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572544.184847:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572544.184869:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35924:x1875453360420224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 283us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.184876:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4298 00010000:00010000:3.0:1788572544.185204:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e368f18/0x85ace67feb1630ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd42 expref: 15 pid: 6681 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.185214:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382364928:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.185233:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.185239:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:3.0:1788572544.185258:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35923:x1875453360419200:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5532us (7961us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.185263:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1157 00000100:00100000:0.0:1788572544.188947:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.189039:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.189153:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420352 00000100:00100000:3.0:1788572544.189163:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420352 00000100:00100000:3.0:1788572544.189223:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420480 00000100:00100000:3.0:1788572544.189227:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420480 00000100:00100000:0.0:1788572544.189261:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1158 00000100:00100000:0.0:1788572544.189273:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360420352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1788572544.189287:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1159 00000100:00100000:1.0:1788572544.189297:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360420480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.189302:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.189304:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.189323:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd5e expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.189323:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd65 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572544.189350:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737f, blocks: 0x1 00002000:00010000:1.0:1788572544.189350:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b737f, blocks: 0x1 00010000:00010000:0.0:1788572544.189367:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=892s v=(0 0) 00010000:00010000:1.0:1788572544.189367:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=892s v=(0 0) 00010000:00010000:0.0:1788572544.189372:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572544.189372:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.189382:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.189382:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.189400:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.189400:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 15 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.189413:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00010000:00010000:1.0:1788572544.189413:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:0.0:1788572544.189646:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360420352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 374us (708us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.189654:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1158 00000100:00100000:2.0:1788572544.189773:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb473800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382365056:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572544.189796:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.189817:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360419968:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5232us (5852us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.189824:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1017 00000100:00100000:1.0:1788572544.190416:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360420480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1120us (1374us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.190423:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1159 00000100:00100000:0.0:1788572544.193948:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.193993:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.194001:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420864 00000100:00100000:3.0:1788572544.194084:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420864 00000100:00100000:3.0:1788572544.194141:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4299 00000100:00100000:3.0:1788572544.194156:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec190c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35925:x1875453360420864:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572544.194198:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420608 00000100:00100000:1.0:1788572544.194203:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420608 00002000:00100000:1.0:1788572544.194234:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4e:0x0] (0->2) req@ffff9627fb471180 x1875453360420608/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572544.194243:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb471180 with x1875453360420608 ext(0->2) 00002000:00010000:1.0:1788572544.194249:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb471180 00000100:00100000:3.0:1788572544.194275:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec190c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35925:x1875453360420864:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 124us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.194284:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4299 00000100:00100000:1.0:1788572544.194405:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 260 00000100:00100000:1.0:1788572544.194413:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360420608:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572544.194850:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2347:x1875453360420608:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 437us (855us total) trans 4294967605 rc 0/0 00000100:00100000:1.0:1788572544.194857:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 260 00002000:00100000:1.0:1788572544.194863:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4e:0x0] (0->2) req@ffff9627fb471180 x1875453360420608/t4294967605(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/448 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572544.194871:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb471180 with x1875453360420608 ext(0->2) 00002000:00010000:1.0:1788572544.194875:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb471180 00000100:00100000:3.0:1788572544.198791:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.198886:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360420992 00000100:00100000:2.0:1788572544.198897:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360420992 00000100:00100000:2.0:1788572544.198967:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4300 00000100:00100000:2.0:1788572544.198978:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35923:x1875453360420992:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.201914:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.201954:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360421248 00000100:00100000:0.0:1788572544.201964:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360421248 00000100:00100000:0.0:1788572544.202030:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4301 00000100:00100000:0.0:1788572544.202043:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35924:x1875453360421248:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.202153:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35924:x1875453360421248:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 113us (248us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.202161:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4301 00000100:00100000:3.0:1788572544.211680:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.211758:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.211786:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360421376 00000100:00100000:3.0:1788572544.211798:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360421376 00000100:00100000:3.0:1788572544.211843:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360421504 00000100:00100000:3.0:1788572544.211846:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360421504 00000100:00100000:3.0:1788572544.211866:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1160 00000100:00100000:3.0:1788572544.211877:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35925:x1875453360421376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.211897:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.211918:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd6c expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.211952:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.211962:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.211967:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.211973:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.212024:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.212039:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.212055:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382365184:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.212131:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.212149:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1161 00000100:00100000:3.0:1788572544.212157:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35924:x1875453360421504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.212165:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.212176:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd73 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.212194:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.212202:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.212205:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.212209:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.212228:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.212238:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.212249:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382365312:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.212271:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.220338:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382365312:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.220377:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.220400:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572544.220465:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35924:x1875453360421504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8308us (8705us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.220475:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1161 00000100:00100000:3.0:1788572544.220530:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.220656:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.220721:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382365184:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572544.220740:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.220753:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cce00, rc 0) 00000100:00100000:0.0:1788572544.220787:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35925:x1875453360421376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8912us (9121us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.220795:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1160 00000100:00100000:0.0:1788572544.222109:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360421632 00000100:00100000:0.0:1788572544.222119:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360421632 00010000:00010000:0.0:1788572544.222159:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.222207:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360421760 00000100:00100000:0.0:1788572544.222210:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360421760 00010000:00010000:0.0:1788572544.222225:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.222247:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1018 00000100:00100000:0.0:1788572544.222256:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360421632:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.222270:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.222294:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222306:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222328:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb1630ff lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd5e expref: 14 pid: 10416 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222343:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.222348:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.222357:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222368:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222404:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.222419:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382365440:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572544.222500:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1317c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.222520:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1019 00000100:00100000:0.0:1788572544.222528:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2349:x1875453360421760:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.222536:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.222551:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222562:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222578:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094303b96/0x85ace67feb1630f8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd65 expref: 17 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222588:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.222592:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.222599:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222607:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.222629:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.222639:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb473800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382365568:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572544.222663:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.228138:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb473800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382365568:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572544.228168:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.228226:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360421760:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5697us (7568us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.228237:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1019 00000100:00100000:0.0:1788572544.228282:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382365440:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572544.228292:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.228311:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360421632:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6056us (7781us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.228318:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1018 00000100:00100000:2.0:1788572544.239419:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.239464:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.239488:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360421888 00000100:00100000:2.0:1788572544.239496:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360421888 00002000:00100000:2.0:1788572544.239547:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4e:0x0] (0->2) req@ffff9627e7849c00 x1875453360421888/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.239557:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7849c00 with x1875453360421888 ext(0->2) 00002000:00010000:2.0:1788572544.239563:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7849c00 00000100:00100000:2.0:1788572544.239585:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422016 00000100:00100000:2.0:1788572544.239588:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422016 00002000:00100000:2.0:1788572544.239603:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x4f:0x0] (0->2) req@ffff9627e7848a80 x1875453360422016/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.239610:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7848a80 with x1875453360422016 ext(0->2) 00002000:00010000:2.0:1788572544.239613:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e7848a80 00000100:00100000:2.0:1788572544.239628:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 261 00000100:00100000:2.0:1788572544.239637:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360421888:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.240053:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360421888:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 415us (643us total) trans 4294967609 rc 0/0 00000100:00100000:2.0:1788572544.240061:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 261 00002000:00100000:2.0:1788572544.240067:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4e:0x0] (0->2) req@ffff9627e7849c00 x1875453360421888/t4294967609(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/448 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.240075:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7849c00 with x1875453360421888 ext(0->2) 00002000:00010000:2.0:1788572544.240078:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7849c00 00000100:00100000:2.0:1788572544.240096:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 262 00000100:00100000:2.0:1788572544.240102:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360422016:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.240312:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360422016:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 209us (846us total) trans 4294967606 rc 0/0 00000100:00100000:2.0:1788572544.240319:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 262 00002000:00100000:2.0:1788572544.240325:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x4f:0x0] (0->2) req@ffff9627e7848a80 x1875453360422016/t4294967606(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/448 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.240334:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7848a80 with x1875453360422016 ext(0->2) 00002000:00010000:2.0:1788572544.240337:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e7848a80 00000100:00100000:1.0:1788572544.243886:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.243964:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.244589:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422144 00000100:00100000:2.0:1788572544.244600:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422144 00000100:00100000:2.0:1788572544.244655:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422272 00000100:00100000:2.0:1788572544.244658:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422272 00000100:00100000:2.0:1788572544.244688:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4302 00000100:00100000:2.0:1788572544.244699:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35924:x1875453360422144:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.244829:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35924:x1875453360422144:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 130us (954us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.244837:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4302 00000100:00100000:2.0:1788572544.244858:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4303 00000100:00100000:2.0:1788572544.244866:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35925:x1875453360422272:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.244910:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35925:x1875453360422272:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 45us (946us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.244917:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4303 00000100:00100000:3.0:1788572544.251790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.251829:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.251903:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422400 00000100:00100000:0.0:1788572544.251913:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422400 00000100:00100000:0.0:1788572544.251965:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422528 00000100:00100000:0.0:1788572544.251969:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422528 00000100:00100000:0.0:1788572544.252027:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1162 00000100:00100000:0.0:1788572544.252038:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360422400:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.252066:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1163 00000100:00100000:2.0:1788572544.252081:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360422528:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.264571:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35923:x1875453360420992:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 65584us (65781us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.264585:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4300 00000001:00080000:2.0:1788572544.264629:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970515 is committed 00000004:00080000:2.0:1788572544.264644:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 42 committed 00000020:00080000:2.0:1788572544.264664:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d13170c0 refcount 2 committed 0 result 0 batchid 120 00000020:00080000:2.0:1788572544.264668:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3700 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d13176c0 00000020:00080000:2.0:1788572544.264672:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627d12fbd00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9628020de000 00000020:00080000:2.0:1788572544.264696:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 117 batchid updated 00000020:00080000:2.0:1788572544.264717:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ec075a80 refcount 2 committed 0 result 0 batchid 118 00000020:00080000:2.0:1788572544.264721:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad400 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ec075480 00000020:00080000:2.0:1788572544.264724:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ad800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4037000 00000020:00080000:2.0:1788572544.264754:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d1151600 refcount 2 committed 0 result 0 batchid 119 00000020:00080000:2.0:1788572544.264757:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aef00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d1151780 00000020:00080000:2.0:1788572544.264761:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c47a1000 00000020:00080000:0.0:1788572544.266555:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 120 committed batchid 117 00000020:00080000:0.0:1788572544.266790:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 117 00000100:00100000:2.0:1788572544.269128:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7848e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382365696:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572544.269344:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.269390:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572544.269397:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7849500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382365824:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572544.269419:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.269548:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e784a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382365952:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572544.269571:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.271842:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1788572544.271862:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7849880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382366080:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572544.271922:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572544.271998:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382365696 00000100:00100000:1.0:1788572544.272092:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382365696 00000100:00100000:1.0:1788572544.273181:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382365824 00000100:00100000:1.0:1788572544.273189:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382365824 00000100:00100000:1.0:1788572544.273202:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382365952 00000100:00100000:1.0:1788572544.273205:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382365952 00000100:00100000:1.0:1788572544.273219:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1164 00000100:00100000:1.0:1788572544.273228:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382365696:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572544.273511:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382365696:12345-0@lo:2:osp-syn-0-0.0 Request processed in 284us (4310us total) trans 4294967607 rc 0/0 00000100:00100000:1.0:1788572544.273519:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1164 00000100:00100000:1.0:1788572544.273535:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1165 00000100:00100000:1.0:1788572544.273544:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382365824:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572544.273629:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382365824:12345-0@lo:2:osp-syn-0-0.0 Request processed in 88us (4209us total) trans 4294967608 rc 0/0 00000100:00100000:1.0:1788572544.273635:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1165 00000100:00100000:1.0:1788572544.273646:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1166 00000100:00100000:1.0:1788572544.273655:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3304:x1875453382365952:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572544.273741:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382365952:12345-0@lo:2:osp-syn-0-0.0 Request processed in 88us (4168us total) trans 4294967609 rc 0/0 00000100:00100000:1.0:1788572544.273747:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1166 00000004:00080000:2.0:1788572544.276652:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e784a680/2, rc 0, transno 4294967609 00000100:00100000:2.0:1788572544.276671:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e784a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382365952:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572544.276710:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7849500/2, rc 0, transno 4294967608 00000100:00100000:2.0:1788572544.276715:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7849500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382365824:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572544.276725:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7848e00/2, rc 0, transno 4294967607 00000100:00100000:2.0:1788572544.276728:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7848e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382365696:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572544.278657:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.280850:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382366080 00000100:00100000:3.0:1788572544.280864:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382366080 00000100:00100000:3.0:1788572544.280939:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422656 00000100:00100000:3.0:1788572544.280942:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422656 00000100:00100000:3.0:1788572544.280962:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1167 00000100:00100000:3.0:1788572544.280972:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382366080:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:3.0:1788572544.281443:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572544.281458:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb mtime from disk: 0 -> 1788572543 00002000:00010000:3.0:1788572544.281460:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1788572544.281463:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb ctime from disk: 1788572543 -> 1788572543 00002000:00010000:3.0:1788572544.281465:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572544.281544:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382366080:12345-0@lo:2:osp-syn-0-0.0 Request processed in 573us (9621us total) trans 4294967610 rc 0/0 00000100:00100000:3.0:1788572544.281551:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1167 00000100:00100000:3.0:1788572544.281572:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1168 00000100:00100000:3.0:1788572544.281581:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2347:x1875453360422656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000004:00080000:2.0:1788572544.281671:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7849880/2, rc 0, transno 4294967610 00000100:00100000:2.0:1788572544.281691:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7849880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382366080:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1788572544.290478:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2348:x1875453360422400:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 38430us (38687us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.290493:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1162 00000100:00100000:2.0:1788572544.297755:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.297801:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360422912 00000100:00100000:2.0:1788572544.297811:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360422912 00000100:00100000:2.0:1788572544.297955:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4304 00000100:00100000:2.0:1788572544.297966:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35924:x1875453360422912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.298148:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.298174:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1630f1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd57 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.298192:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:2.0:1788572544.298929:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572544.298965:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.298981:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.298993:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572544.299482:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b4a03470 reply state 000000004c384c75 transno 4294970516 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.299493:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572544.299610:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35924:x1875453360422912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1641us (1864us total) trans 4294970516 rc 0/0 00000100:00100000:2.0:1788572544.299620:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4304 00000001:00080000:3.0:1788572544.305845:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967604 is committed 00000001:00080000:3.0:1788572544.305865:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967602 is committed 00000001:00080000:3.0:1788572544.305872:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967605 is committed 00000001:00080000:3.0:1788572544.305884:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967606 is committed 00000100:00100000:2.0:1788572544.307983:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572544.308110:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.308125:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.308138:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.308145:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.308150:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.308160:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.308177:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:1.0:1788572544.308181:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.308186:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.308193:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163114 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.308329:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360423040 00000100:00100000:3.0:1788572544.308339:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360423040 00000100:00100000:3.0:1788572544.308387:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1169 00000100:00100000:3.0:1788572544.308397:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360423040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572544.308646:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572544.308650:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:3.0:1788572544.308653:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:3.0:1788572544.308655:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb ctime from disk: 1788572543 -> 1788572527 00002000:00010000:3.0:1788572544.308658:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572544.308717:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360423040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 321us (742us total) trans 4294967611 rc 0/0 00000100:00100000:3.0:1788572544.308725:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1169 00000100:00100000:0.0:1788572544.320849:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.323374:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360423296 00000100:00100000:3.0:1788572544.323388:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360423296 00000100:00100000:3.0:1788572544.323462:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4305 00000100:00100000:3.0:1788572544.323474:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35924:x1875453360423296:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.323545:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.323565:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.323581:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.323617:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.323632:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 65 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.323648:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382366208:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.323744:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.332360:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.333244:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360423424 00000100:00100000:2.0:1788572544.333256:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360423424 00010000:00010000:2.0:1788572544.333297:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 66 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.333340:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1020 00000100:00100000:2.0:1788572544.333351:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360423424:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.333365:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.333372:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 66 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.333387:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 66 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.333410:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:2.0:1788572544.333415:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.333426:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.333435:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.333465:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.333516:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360423424:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 167us (1164us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.333522:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1020 00010000:00010000:1.0:1788572544.336354:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb1630a4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd26 expref: 65 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.336381:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382366208:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.336405:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336415:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336421:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336469:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336477:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336485:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336492:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.336496:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336504:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336521:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:1.0:1788572544.336524:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572544.336530:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.336536:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16311b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.336608:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35924:x1875453360423296:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13135us (15768us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572544.336618:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4305 00000100:00100000:2.0:1788572544.340934:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360422528:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 88843us (89087us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.340949:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1163 00000001:00080000:1.0:1788572544.341503:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967608 is committed 00000001:00080000:1.0:1788572544.341516:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967609 is committed 00000001:00080000:1.0:1788572544.341527:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967606 is committed 00000001:00080000:1.0:1788572544.341538:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967607 is committed 00000100:00100000:0.0:1788572544.353979:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.354159:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360423552 00000100:00100000:1.0:1788572544.354170:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360423552 00000100:00100000:1.0:1788572544.354252:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1170 00000100:00100000:1.0:1788572544.354264:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360423552:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.354311:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.354331:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.354341:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572544.354345:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572544.354351:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.354387:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.354402:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.354417:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382366336:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.354506:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.354652:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.354865:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360423808 00000100:00100000:1.0:1788572544.354871:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360423808 00000100:00100000:1.0:1788572544.354918:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4306 00000100:00100000:1.0:1788572544.354928:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35925:x1875453360423808:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.354974:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.354996:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1630e3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd50 expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.355407:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:1.0:1788572544.355412:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.355444:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.355469:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.355479:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572544.355650:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cd135141 reply state 000000003a2c2713 transno 4294970517 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.355659:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572544.355839:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35925:x1875453360423808:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 910us (1185us total) trans 4294970517 rc 0/0 00000100:00100000:1.0:1788572544.355857:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4306 00000001:00080000:2.0:1788572544.358520:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967609 is committed 00000001:00080000:2.0:1788572544.358544:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967610 is committed 00000100:00100000:3.0:1788572544.360989:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360422656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 79399us (82330us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.361032:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1168 00000100:00100000:1.0:1788572544.362433:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382366336:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.362490:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.362569:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.362660:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360423936 00000100:00100000:3.0:1788572544.362668:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360423936 00010000:00010000:3.0:1788572544.362701:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.362741:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1021 00000100:00100000:3.0:1788572544.362750:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360423936:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572544.362764:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.362786:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.362807:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.362830:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb16310d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd73 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.362844:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572544.362860:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572544.362866:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.362876:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.362903:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572544.362952:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360423936:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 202us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.362959:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1021 00010000:00010000:3.0:1788572544.362982:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.362991:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363036:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363046:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363054:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363061:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.363065:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363074:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363085:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.363093:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d8507034/0x85ace67feb163122 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.363135:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360423552:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 8874us (9166us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.363142:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1170 00000100:00100000:0.0:1788572544.369169:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.369217:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.369244:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.369272:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360424064 00000100:00100000:0.0:1788572544.369281:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360424064 00000100:00100000:0.0:1788572544.369333:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1171 00000100:00100000:0.0:1788572544.369344:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360424064:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572544.369612:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4e:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572544.369616:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4e:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:0.0:1788572544.369619:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4e:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:0.0:1788572544.369621:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4e:0x0] updating lvb ctime from disk: 1788572543 -> 1788572527 00002000:00010000:0.0:1788572544.369624:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4e:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572544.369680:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360424064:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 338us (462us total) trans 4294967610 rc 0/0 00000100:00100000:0.0:1788572544.369689:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1171 00010000:00010000:2.0:1788572544.369995:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.370112:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.370121:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.370127:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.370132:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.370140:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.370155:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:2.0:1788572544.370158:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.370163:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.370170:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163129 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.370259:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360424320 00000100:00100000:2.0:1788572544.370264:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360424320 00000100:00100000:2.0:1788572544.370335:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360424576 00000100:00100000:2.0:1788572544.370339:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360424576 00000100:00100000:3.0:1788572544.370478:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4307 00000100:00100000:3.0:1788572544.370490:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35924:x1875453360424320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.370566:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370582:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370592:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370779:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370791:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 66 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370799:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370831:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 66 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.370861:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 66 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.370877:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0c700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382366464:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.371144:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3251480 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.371435:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4308 00000100:00100000:2.0:1788572544.371456:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35923:x1875453360424576:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.371492:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.371510:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1630b9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd34 expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.371523:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:2.0:1788572544.371528:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572544.371547:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.371556:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.371564:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572544.371660:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d90952aa reply state 00000000f245f42f transno 4294970519 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.371671:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572544.371733:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35923:x1875453360424576:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 290us (2487us total) trans 4294970519 rc 0/0 00000100:00100000:2.0:1788572544.371741:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4308 00000100:00100000:2.0:1788572544.381987:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.382051:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.382079:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.382105:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360424704 00000100:00100000:2.0:1788572544.382115:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360424704 00010000:00010000:2.0:1788572544.382154:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 67 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.382192:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1022 00000100:00100000:2.0:1788572544.382213:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec085450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2347:x1875453360424704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.382232:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.382239:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 67 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382255:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 67 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382280:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:2.0:1788572544.382285:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382295:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382302:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382324:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.382373:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec085450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2347:x1875453360424704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 173us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.382379:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1022 00000100:00100000:2.0:1788572544.382429:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360424832 00000100:00100000:2.0:1788572544.382433:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360424832 00000100:00100000:2.0:1788572544.382471:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1172 00000100:00100000:2.0:1788572544.382479:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360424832:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572544.382681:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572544.382685:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb mtime from disk: 1788572543 -> 1788572517 00002000:00010000:2.0:1788572544.382688:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:2.0:1788572544.382691:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb ctime from disk: 1788572543 -> 1788572527 00002000:00010000:2.0:1788572544.382693:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4e:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572544.382716:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360424832:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 238us (665us total) trans 4294967612 rc 0/0 00000100:00100000:2.0:1788572544.382723:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1172 00000100:00100000:2.0:1788572544.382746:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360425088 00000100:00100000:2.0:1788572544.382749:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360425088 00000100:00100000:2.0:1788572544.382772:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4309 00000100:00100000:2.0:1788572544.382778:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35925:x1875453360425088:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.382823:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382835:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382843:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382893:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.382906:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 67 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.382920:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382366592:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.382967:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56840 going to sleep for 11 seconds 00010000:00010000:2.0:1788572544.383029:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb1630c7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd3b expref: 67 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.383049:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0c700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382366464:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572544.383067:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.383075:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.383081:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572544.383218:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970518 00000001:00080000:2.0:1788572544.383355:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970520 00000001:00080000:2.0:1788572544.383401:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970521 00000100:00100000:2.0:1788572544.383478:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382366720:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572544.383525:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.383556:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8ff5f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.384633:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382366720 00000100:00100000:2.0:1788572544.384643:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382366720 00000100:00100000:2.0:1788572544.384701:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:2.0:1788572544.384711:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382366720:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572544.384810:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382366720:12345-0@lo:1000:mdt00_000.0 Request processed in 100us (1285us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.384818:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:2.0:1788572544.384864:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967610 gen 2 00000004:00080000:2.0:1788572544.384871:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967602 committed transno 4294967610 00000004:00080000:2.0:1788572544.384889:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e7848e00, transno 4294967607 00000004:00080000:2.0:1788572544.384899:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e7849500, transno 4294967608 00000004:00080000:2.0:1788572544.384902:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e784a680, transno 4294967609 00000004:00080000:2.0:1788572544.384904:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e7849880, transno 4294967610 00000100:00100000:2.0:1788572544.384910:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382366720:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1788572544.385181:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.385193:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.385211:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.385217:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.385223:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.385232:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.385249:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:1.0:1788572544.385252:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.385257:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.385263:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16313e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572544.385917:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970522 00000004:00080000:0.0:1788572544.389370:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:1.0:1788572544.389743:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.389772:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.389844:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382366848:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572544.390142:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382366592:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572544.390167:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.390332:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360425472 00000100:00100000:2.0:1788572544.390340:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360425472 00000100:00100000:2.0:1788572544.390427:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4310 00000100:00100000:2.0:1788572544.390438:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35923:x1875453360425472:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.390515:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.390532:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.390540:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.390572:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.390584:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 70 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.390598:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382366976:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.390676:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.390717:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.390740:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb0db240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.390835:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360425216 00000100:00100000:3.0:1788572544.390840:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360425216 00010000:00010000:3.0:1788572544.390876:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 71 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.390912:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1023 00000100:00100000:3.0:1788572544.391166:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2349:x1875453360425216:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572544.391180:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.391187:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 71 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.391270:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 71 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.391291:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163081 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd1f expref: 71 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.391306:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.391311:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.391319:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.391325:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.391355:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572544.391385:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360425216:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 466us (1647us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.391393:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1023 00010000:00010000:2.0:1788572544.391531:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391539:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391566:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391573:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391580:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391586:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.391590:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391597:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391606:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:2.0:1788572544.391609:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572544.391613:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.391619:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163145 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.391654:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35925:x1875453360425088:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8876us (9574us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572544.391665:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4309 00000100:00100000:1.0:1788572544.395345:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382366848 00000100:00100000:1.0:1788572544.395361:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382366848 00000100:00100000:1.0:1788572544.395435:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:1.0:1788572544.395458:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382366848:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572544.395752:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382366848:12345-0@lo:1000:mdt00_000.0 Request processed in 306us (5032us total) trans 4294967613 rc 0/0 00000100:00100000:1.0:1788572544.395764:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00100000:0.0:1788572544.396294:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.396429:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382366976:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.396470:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.396557:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360425600 00000100:00100000:1.0:1788572544.396563:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360425600 00000100:00100000:1.0:1788572544.396603:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1173 00000100:00100000:1.0:1788572544.396613:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360425600:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.396647:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.396666:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.396675:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572544.396680:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572544.396685:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.396719:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.396733:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 11 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.396748:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382367104:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.396818:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.397345:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382366848:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1788572544.397385:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397393:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397399:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397405:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.397410:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397417:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397432:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716d00) 00010000:00010000:3.0:1788572544.397435:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.397440:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397459:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163137 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x8f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397491:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397498:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397504:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397510:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.397513:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397519:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397527:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572544.397530:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.397533:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397538:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb15f72a lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397555:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397566:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397574:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397580:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397586:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397592:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397597:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.397599:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397605:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397610:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572544.397613:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.397616:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397621:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16315a lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397629:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397634:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397640:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397644:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.397647:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397656:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397662:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572544.397664:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.397666:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.397671:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163130 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.397894:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35924:x1875453360424320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 27405us (28734us total) trans 4294970523 rc 0/0 00000100:00100000:3.0:1788572544.397904:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4307 00000020:00080000:3.0:1788572544.397909:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572544.403550:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.403666:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360425728 00000100:00100000:2.0:1788572544.403677:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360425728 00010000:00010000:2.0:1788572544.403722:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 68 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.403770:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1024 00000100:00100000:2.0:1788572544.403781:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360425728:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.403796:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.403803:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 68 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.403816:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 68 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.403839:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb16303b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x8d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd03 expref: 68 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.403856:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:2.0:1788572544.403862:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.403873:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.403881:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.403926:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.403973:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360425728:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 195us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.403981:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1024 00000100:00100000:1.0:1788572544.404195:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382367104:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.404247:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404461:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404470:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404501:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404508:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404516:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404522:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.404527:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404535:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404546:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:0.0:1788572544.404549:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572544.404554:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.404561:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16314c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.404619:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35923:x1875453360425472:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 14182us (14845us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572544.404628:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4310 00000100:00100000:0.0:1788572544.407370:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.407414:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.407482:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360425856 00000100:00100000:2.0:1788572544.407492:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360425856 00010000:00010000:2.0:1788572544.407534:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.407577:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1025 00000100:00100000:2.0:1788572544.407586:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360425856:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.407600:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.407623:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.407637:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.407659:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb163106 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd6c expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.407677:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572544.407682:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572544.407688:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.407704:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.407740:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.407790:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360425856:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 204us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.407798:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1025 00010000:00010000:0.0:1788572544.407946:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.407957:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.407983:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.407992:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.407999:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.408039:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.408044:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.408054:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.408067:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.408075:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8c9164c/0x85ace67feb163153 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.408135:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360425600:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11523us (11843us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.408144:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1173 00000100:00100000:1.0:1788572544.408630:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360426112 00000100:00100000:1.0:1788572544.408636:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360426112 00000100:00100000:1.0:1788572544.408693:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 917 00000100:00100000:1.0:1788572544.408703:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35924:x1875453360426112:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572544.409495:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x8f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572544.409600:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970524 00010000:00010000:1.0:1788572544.409645:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.409660:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.409670:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.409678:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.409685:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.409690:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.409709:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788572544.410026:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572544.410033:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.410040:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572544.410044:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.410052:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.410064:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8f:0x0].0x0 (ffff9627c5716b00) 00010000:00010000:2.0:1788572544.410067:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572544.410071:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.410078:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163161 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.410094:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572544.410096:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572544.410098:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572544.410193:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35924:x1875453360426112:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1490us (2830us total) trans 4294970525 rc 0/0 00000100:00100000:1.0:1788572544.410206:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 917 00000020:00080000:1.0:1788572544.410211:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788572544.414570:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.414646:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360426240 00000100:00100000:0.0:1788572544.414656:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360426240 00000100:00100000:0.0:1788572544.414718:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1174 00000100:00100000:0.0:1788572544.414728:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360426240:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.414773:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.414795:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 12 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.414803:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.414808:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.414812:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.414862:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 12 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.414877:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 12 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.414893:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ca8c8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382367232:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572544.414985:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.420865:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ca8c8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382367232:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572544.420898:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.424623:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.424716:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360426368 00000100:00100000:3.0:1788572544.424727:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360426368 00010000:00010000:3.0:1788572544.424772:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 13 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.424818:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1026 00000100:00100000:3.0:1788572544.424828:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360426368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572544.424858:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.424885:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 13 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.424900:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 13 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.424928:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb32d297/0x85ace67feb1630d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd49 expref: 13 pid: 6679 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.424945:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572544.424949:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572544.424955:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.424966:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.424998:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572544.425086:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360426368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 258us (472us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.425094:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1026 00010000:00010000:0.0:1788572544.425295:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425306:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425331:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425339:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425352:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425359:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.425364:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425374:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425386:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.425394:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb163168 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.425440:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360426240:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 10714us (10881us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.425459:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1174 00000100:00100000:2.0:1788572544.431000:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.431290:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360426624 00000100:00100000:1.0:1788572544.431302:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360426624 00000100:00100000:1.0:1788572544.431379:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4311 00000100:00100000:1.0:1788572544.431391:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35925:x1875453360426624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.431491:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431508:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431519:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431769:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431782:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431791:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431827:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 69 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.431841:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 69 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.431864:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382367360:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.431977:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.435086:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.435299:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360426880 00000100:00100000:1.0:1788572544.435311:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360426880 00000100:00100000:1.0:1788572544.435387:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4312 00000100:00100000:1.0:1788572544.435399:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35923:x1875453360426880:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.435493:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435512:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435525:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435765:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435778:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 71 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435786:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435823:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 71 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.435837:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 71 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.435852:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382367488:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.436139:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1151780 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.444765:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.444906:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360427008 00000100:00100000:0.0:1788572544.444917:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360427008 00010000:00010000:0.0:1788572544.444961:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 72 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.445088:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382367360:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.445131:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.445221:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1027 00000100:00100000:0.0:1788572544.445234:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2350:x1875453360427008:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.445250:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.445257:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 72 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.445270:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 72 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.445293:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1630ab lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd2d expref: 72 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.445308:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:0.0:1788572544.445313:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.445323:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.445330:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.445353:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.445402:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2350:x1875453360427008:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 171us (648us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.445410:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1027 00010000:00010000:3.0:1788572544.445469:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.445476:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572544.445671:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970526 00000001:00080000:3.0:1788572544.445839:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970528 00000001:00080000:3.0:1788572544.445907:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970529 00000100:00100000:3.0:1788572544.445989:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382367616:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572544.446069:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572544.446092:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.447122:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382367488:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.447159:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.447357:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.448743:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360427136 00000100:00100000:2.0:1788572544.448757:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360427136 00010000:00010000:2.0:1788572544.448802:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 74 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.448843:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1028 00000100:00100000:1.0:1788572544.448875:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2350:x1875453360427136:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572544.448889:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.448897:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 74 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.448910:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 74 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.448931:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16306c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd18 expref: 74 pid: 5837 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.448949:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:1.0:1788572544.448954:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.448962:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.448970:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.449164:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.449175:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572544.449363:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970527 00000001:00080000:3.0:1788572544.449536:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970531 00000001:00080000:3.0:1788572544.449586:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970532 00000100:00100000:3.0:1788572544.449671:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382367744:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572544.449732:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572544.449764:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f64600 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.449787:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382367616 00000100:00100000:3.0:1788572544.449792:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382367616 00000100:00100000:3.0:1788572544.449824:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382367744 00000100:00100000:3.0:1788572544.449827:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382367744 00000100:00100000:3.0:1788572544.449841:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:3.0:1788572544.449847:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382367616:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572544.449920:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382367616:12345-0@lo:1000:mdt00_003.0 Request processed in 73us (3851us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.449928:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:3.0:1788572544.449945:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:3.0:1788572544.449951:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382367744:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572544.450000:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382367744:12345-0@lo:1000:mdt00_001.0 Request processed in 48us (267us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.450040:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:3.0:1788572544.450080:0:9040:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967607 gen 2 00000004:00080000:3.0:1788572544.450086:0:9040:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967606 committed transno 4294967607 00000004:00080000:3.0:1788572544.450105:0:9040:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967607 committed transno 4294967607 00000100:00100000:3.0:1788572544.450120:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382367616:0@lo:1000:mdt00_003.0 00000001:00080000:3.0:1788572544.450511:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970530 00000100:00100000:3.0:1788572544.450568:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382367872:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572544.450597:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572544.450615:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.450628:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382367872 00000100:00100000:3.0:1788572544.450632:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382367872 00000100:00100000:3.0:1788572544.450654:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1788572544.450659:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382367872:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572544.450810:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382367872:12345-0@lo:1000:mdt00_003.0 Request processed in 150us (211us total) trans 4294967611 rc 0/0 00000100:00100000:3.0:1788572544.450818:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1788572544.450852:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382367872:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1788572544.450881:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450891:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450898:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450908:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.450913:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450920:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450935:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.450939:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.450943:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450950:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb163176 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x8e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450983:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450989:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.450995:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451000:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.451023:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451030:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451042:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:3.0:1788572544.451044:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.451047:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451053:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb15f7a1 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451077:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451090:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451099:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451106:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451112:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451119:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451124:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.451127:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451132:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451139:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:3.0:1788572544.451141:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.451144:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451150:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16318b lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451158:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451164:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451170:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451175:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.451178:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451184:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451191:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:3.0:1788572544.451193:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.451196:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.451202:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16316f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.451351:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35925:x1875453360426624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 19962us (20359us total) trans 4294970534 rc 0/0 00000100:00100000:3.0:1788572544.451360:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4311 00000020:00080000:3.0:1788572544.451363:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572544.451397:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382367744:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1788572544.451669:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970533 00000100:00100000:3.0:1788572544.451722:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382368000:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572544.451757:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572544.451775:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f64000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.451792:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382368000 00000100:00100000:3.0:1788572544.451796:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382368000 00000100:00100000:3.0:1788572544.451823:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1788572544.451830:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382368000:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572544.451973:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:5838:x1875453382368000:12345-0@lo:1000:mdt00_001.0 Request processed in 144us (216us total) trans 4294967614 rc 0/0 00000100:00100000:3.0:1788572544.451981:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1788572544.452033:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382368000:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1788572544.452056:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452063:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452071:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452077:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.452082:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452089:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452099:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:3.0:1788572544.452102:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.452106:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452113:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163184 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x8d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452130:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452136:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452141:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452145:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478ee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.452148:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452152:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452158:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627c5716400) 00010000:00010000:3.0:1788572544.452160:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.452163:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452168:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb15f6c8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452184:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452194:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452201:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452207:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452213:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452218:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452223:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.452225:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452230:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452237:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627c5716400) 00010000:00010000:3.0:1788572544.452239:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.452242:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452247:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163192 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452257:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452263:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452269:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452273:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.452275:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452281:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452288:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627c5716400) 00010000:00010000:3.0:1788572544.452291:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.452294:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.452299:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16317d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.452411:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35923:x1875453360426880:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 17013us (17333us total) trans 4294970535 rc 0/0 00000100:00100000:3.0:1788572544.452420:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4312 00000020:00080000:3.0:1788572544.452424:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00010000:00010000:1.0:1788572544.452595:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572544.452624:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2350:x1875453360427136:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3762us (5267us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.452633:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1028 00000100:00100000:1.0:1788572544.455562:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.455608:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.455712:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360427392 00000100:00100000:0.0:1788572544.455723:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360427392 00000100:00100000:0.0:1788572544.455774:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360427648 00000100:00100000:0.0:1788572544.455777:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360427648 00000100:00100000:0.0:1788572544.455795:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 918 00000100:00100000:0.0:1788572544.455804:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35923:x1875453360427392:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572544.456228:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x8d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572544.456273:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970536 00010000:00010000:0.0:1788572544.456320:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456336:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456349:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456358:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456365:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456372:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456391:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572544.456492:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35923:x1875453360427392:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 687us (938us total) trans 4294970537 rc 0/0 00000100:00100000:0.0:1788572544.456503:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 918 00000020:00080000:0.0:1788572544.456509:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788572544.456559:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 919 00000100:00100000:0.0:1788572544.456568:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35925:x1875453360427648:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572544.456730:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x8e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572544.456755:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970538 00010000:00010000:0.0:1788572544.456786:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456796:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456804:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456810:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456816:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456822:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456838:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1788572544.456874:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35925:x1875453360427648:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 307us (1265us total) trans 4294970539 rc 0/0 00000100:00100000:0.0:1788572544.456880:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 919 00000020:00080000:0.0:1788572544.456884:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572544.456903:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1788572544.456910:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456917:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.456921:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456934:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456945:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8d:0x0].0x0 (ffff9627e7451b00) 00010000:00010000:0.0:1788572544.456948:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572544.456952:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456958:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb163199 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456976:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572544.456979:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456985:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.456988:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.456994:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.457001:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x8e:0x0].0x0 (ffff9627e7451100) 00010000:00010000:0.0:1788572544.457030:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572544.457034:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.457040:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631a0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x8e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.457050:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.457052:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.457054:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.457060:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1788572544.461430:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572544.461440:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572544.461497:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 148 count 101 00000040:00080000:3.0:1788572544.461504:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:3.0:1788572544.461659:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff5500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382368128:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572544.461939:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.467135:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.467249:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.467601:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff5500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382368128:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572544.469985:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360427776 00000100:00100000:2.0:1788572544.469998:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360427776 00000100:00100000:2.0:1788572544.470082:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360427904 00000100:00100000:2.0:1788572544.470085:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360427904 00000100:00100000:2.0:1788572544.470111:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4313 00000100:00100000:2.0:1788572544.470120:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35926:x1875453360427776:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572544.470163:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb7:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572544.470297:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35926:x1875453360427776:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 177us (3172us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.470305:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4313 00000100:00100000:2.0:1788572544.470325:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4314 00000100:00100000:2.0:1788572544.470332:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35927:x1875453360427904:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572544.470345:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb2:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572544.470383:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35927:x1875453360427904:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 52us (3135us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.470390:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4314 00000100:00100000:2.0:1788572544.470466:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.470579:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360428032 00000100:00100000:2.0:1788572544.470583:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360428032 00000100:00100000:2.0:1788572544.470608:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4315 00000100:00100000:2.0:1788572544.470615:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35928:x1875453360428032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572544.470629:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6c:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572544.470784:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35928:x1875453360428032:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 168us (335us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.470792:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4315 00000100:00100000:2.0:1788572544.474943:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.474986:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.475029:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.475056:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360428160 00000100:00100000:2.0:1788572544.475065:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360428160 00000100:00100000:2.0:1788572544.475115:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360428288 00000100:00100000:2.0:1788572544.475119:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360428288 00000100:00100000:2.0:1788572544.475131:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360428416 00000100:00100000:2.0:1788572544.475134:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360428416 00000100:00100000:2.0:1788572544.475147:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4316 00000100:00100000:2.0:1788572544.475156:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35927:x1875453360428160:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.475232:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475250:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475263:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475278:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475300:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475308:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475336:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475344:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475351:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.475388:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb2:0x0]: returning size 3 00000004:00200000:2.0:1788572544.475392:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.475405:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475417:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475424:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475431:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475437:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.475442:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475478:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475492:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.475495:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.475499:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475505:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1631a7 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475520:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475526:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475532:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475538:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.475541:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475547:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475554:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x78b74cba (ffff9627e7a95e00) 00010000:00010000:2.0:1788572544.475556:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.475559:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475568:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1631ae lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x78b74cba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475584:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475590:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475596:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475602:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.475604:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475609:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475616:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:2.0:1788572544.475619:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.475622:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475628:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1631b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.475698:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35927:x1875453360428160:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 542us (765us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.475706:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4316 00000100:00100000:2.0:1788572544.475727:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4317 00000100:00100000:2.0:1788572544.475734:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35926:x1875453360428288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.475761:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475770:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475777:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475790:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475797:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475817:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475831:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475838:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475844:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.475865:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb7:0x0]: returning size 3 00000004:00200000:2.0:1788572544.475868:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.475874:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475881:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475887:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475893:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475897:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.475900:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475906:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475913:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.475916:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.475918:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475924:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1631bc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475933:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475939:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475944:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475950:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.475953:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475959:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475967:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x23a2211e (ffff9627e7a95f00) 00010000:00010000:2.0:1788572544.475969:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.475972:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475977:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1631c3 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x23a2211e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475988:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475994:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.475999:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476034:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.476037:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476043:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476054:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:2.0:1788572544.476056:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.476059:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476066:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1631ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.476102:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35926:x1875453360428288:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 369us (1115us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.476109:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4317 00000100:00100000:2.0:1788572544.476123:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4318 00000100:00100000:2.0:1788572544.476131:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35928:x1875453360428416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.476155:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476165:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476172:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476182:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476188:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476194:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476210:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476216:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476222:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.476240:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6c:0x0]: returning size 3 00000004:00200000:2.0:1788572544.476243:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.476249:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476256:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476261:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476266:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476270:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.476273:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476278:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476285:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.476288:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.476291:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476295:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1631d1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476304:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476310:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476315:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476320:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.476323:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476328:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476335:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x9bb35311 (ffff9627e7a95a00) 00010000:00010000:2.0:1788572544.476337:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.476339:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476344:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1631d8 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x9bb35311 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476355:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476361:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476366:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476371:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.476374:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476379:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476387:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572544.476389:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.476392:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.476397:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1631df lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.476429:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35928:x1875453360428416:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 299us (1398us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.476435:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4318 00000100:00100000:2.0:1788572544.481189:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.481304:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360428672 00000100:00100000:0.0:1788572544.481318:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360428672 00000100:00100000:0.0:1788572544.481392:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4319 00000100:00100000:0.0:1788572544.481402:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35928:x1875453360428672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.481423:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.481445:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631e6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd7a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481761:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481778:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481790:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.481813:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6c:0x0]: returning size 3 00010000:00010000:0.0:1788572544.481917:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481926:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481933:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481938:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.481943:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481951:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481964:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627c3dbed00) 00010000:00010000:0.0:1788572544.481967:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.481972:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.481979:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb1631ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.482526:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631e6 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd7a expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.482541:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572544.482543:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572544.482548:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb1631e6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd7a expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.482557:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:0.0:1788572544.482644:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35928:x1875453360428672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1242us (1466us total) trans 4294970540 rc 301/301 00000100:00100000:0.0:1788572544.482652:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4319 00000100:00100000:1.0:1788572544.483920:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.484158:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360429184 00000100:00100000:0.0:1788572544.484168:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360429184 00000100:00100000:0.0:1788572544.484230:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4320 00000100:00100000:0.0:1788572544.484242:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35926:x1875453360429184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.484264:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.484282:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb1631fb lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd88 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484394:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484409:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484421:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.484432:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb7:0x0]: returning size 3 00010000:00010000:0.0:1788572544.484555:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484564:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484571:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484577:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.484582:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484590:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484602:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:0.0:1788572544.484606:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.484610:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484616:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb163202 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484654:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb1631fb lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd88 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484662:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572544.484665:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572544.484670:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47800/0x85ace67feb1631fb lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd88 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.484679:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:0.0:1788572544.484727:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35926:x1875453360429184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 489us (813us total) trans 4294970541 rc 301/301 00000100:00100000:0.0:1788572544.484735:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4320 00000100:00100000:1.0:1788572544.485829:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.489922:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.490270:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360428928 00000100:00100000:3.0:1788572544.490280:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360428928 00000100:00100000:0.0:1788572544.490424:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360429312 00000100:00100000:0.0:1788572544.490429:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360429312 00000100:00100000:0.0:1788572544.490480:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4321 00000100:00100000:0.0:1788572544.490490:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35927:x1875453360428928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.490518:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.490540:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163210 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd81 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490645:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490660:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490684:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.490696:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb2:0x0]: returning size 3 00010000:00010000:0.0:1788572544.490796:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490818:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490826:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490832:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.490837:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490854:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490867:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:0.0:1788572544.490871:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.490876:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490882:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163217 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490930:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163210 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd81 expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490939:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572544.490941:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572544.490945:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163210 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd81 expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.490960:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572544.491044:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35927:x1875453360428928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 555us (5220us total) trans 4294970542 rc 301/301 00000100:00100000:0.0:1788572544.491052:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4321 00000100:00100000:0.0:1788572544.491073:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4322 00000100:00100000:0.0:1788572544.491080:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35926:x1875453360429312:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.491097:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb7:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.491165:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35926:x1875453360429312:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 85us (1368us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.491171:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4322 00000100:00100000:1.0:1788572544.493183:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.493233:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.493457:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360429440 00000100:00100000:0.0:1788572544.493465:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360429440 00000100:00100000:0.0:1788572544.493525:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360429568 00000100:00100000:0.0:1788572544.493528:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360429568 00000100:00100000:0.0:1788572544.493553:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4323 00000100:00100000:0.0:1788572544.493562:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35928:x1875453360429440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.493593:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6c:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.493698:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35928:x1875453360429440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 137us (521us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.493706:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4323 00000100:00100000:0.0:1788572544.493722:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4324 00000100:00100000:0.0:1788572544.493729:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35926:x1875453360429568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.493738:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.493755:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163225 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd8f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.493817:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.493831:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.493842:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.493857:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.493865:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.493871:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494190:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16323a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494200:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16323a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494209:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16323a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.494230:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:0.0:1788572544.494234:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572544.494252:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16323a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494262:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494269:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494276:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494282:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.494540:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494550:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494562:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572544.494566:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.494571:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494578:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb16322c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494598:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494605:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494611:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494617:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.494620:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494626:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494633:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627e7451100) 00010000:00010000:0.0:1788572544.494635:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.494638:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494645:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163233 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494690:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163225 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd8f expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494708:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16323a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd8f expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.494721:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572544.494724:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.494727:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572544.495051:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35926:x1875453360429568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1320us (1815us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.495059:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4324 00000100:00100000:2.0:1788572544.503732:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.503834:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.504625:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360429696 00000100:00100000:3.0:1788572544.504636:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360429696 00000100:00100000:0.0:1788572544.504765:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360429824 00000100:00100000:0.0:1788572544.504770:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360429824 00000100:00100000:0.0:1788572544.504837:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4325 00000100:00100000:0.0:1788572544.504854:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35928:x1875453360429824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.504882:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.504903:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163241 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd96 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.504979:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.504995:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505034:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505053:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505060:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505068:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505942:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163256 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505957:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163256 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.505967:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163256 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.505997:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:0.0:1788572544.506001:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572544.506051:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163256 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506064:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506071:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506078:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506084:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.506088:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506095:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506108:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572544.506111:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.506115:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506121:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163248 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506138:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506144:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506149:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506153:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.506156:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506161:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506168:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627e7451b00) 00010000:00010000:0.0:1788572544.506171:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.506173:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506177:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16324f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506444:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163241 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd96 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506482:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd96 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.506491:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572544.506494:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.506497:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:0.0:1788572544.506575:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35928:x1875453360429824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1730us (2740us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.506583:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4325 00000100:00100000:2.0:1788572544.507968:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4326 00000100:00100000:2.0:1788572544.507984:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35927:x1875453360429696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572544.508045:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xb2:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572544.509083:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35927:x1875453360429696:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 1097us (5356us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.509094:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4326 00000100:00100000:2.0:1788572544.509817:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.511573:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360430080 00000100:00100000:1.0:1788572544.511587:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360430080 00000100:00100000:1.0:1788572544.511679:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4327 00000100:00100000:1.0:1788572544.511690:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35926:x1875453360430080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.511712:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.511715:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572544.511744:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16323a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd8f expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.511769:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.511775:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.511790:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16325d lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efd9d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572544.512437:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4f:0x0], index 1: 0 00010000:00010000:1.0:1788572544.512653:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163264 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.512668:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163264 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.512678:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163264 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.512731:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16325d lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd9d expref: 87 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.512747:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163264 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd9d expref: 86 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.512755:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627c5716900) 00010000:00010000:1.0:1788572544.512758:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572544.512761:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96a00, rc 0) 00000100:00100000:1.0:1788572544.512832:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35926:x1875453360430080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1144us (3028us total) trans 4294970543 rc 0/0 00000100:00100000:1.0:1788572544.512841:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4327 00000100:00100000:1.0:1788572544.514532:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.514811:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360430464 00000100:00100000:3.0:1788572544.514820:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360430464 00000100:00100000:3.0:1788572544.514873:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4328 00000100:00100000:3.0:1788572544.514883:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35927:x1875453360430464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.514901:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.515106:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163272 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdab expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515177:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515191:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515271:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515354:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515364:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515371:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515789:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163287 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515799:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163287 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.515874:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163287 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.515905:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:3.0:1788572544.515909:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.516466:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163287 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516482:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516490:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516497:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516503:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.516508:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516518:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516533:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572544.516537:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.516542:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516550:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163279 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516565:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516571:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516578:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516584:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.516586:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516593:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516599:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.516602:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.516605:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516616:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163280 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516651:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb163272 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdab expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.516665:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163287 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdab expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.517242:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572544.517245:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.517249:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572544.517870:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35927:x1875453360430464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2983us (3341us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.517881:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4328 00000100:00100000:0.0:1788572544.519895:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.520138:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360430336 00000100:00100000:2.0:1788572544.520149:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360430336 00000100:00100000:2.0:1788572544.520222:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4329 00000100:00100000:2.0:1788572544.520233:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35928:x1875453360430336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.520254:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.520258:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572544.520292:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163256 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd96 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.520317:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572544.520322:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572544.520336:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16328e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efda4 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572544.521120:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x50:0x0], index 0: 0 00010000:00010000:2.0:1788572544.521469:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163295 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.521485:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163295 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.521497:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163295 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.521559:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb16328e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efda4 expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.521576:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163295 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efda4 expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.521585:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:2.0:1788572544.521588:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572544.521591:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05a00, rc 0) 00000100:00100000:2.0:1788572544.521646:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35928:x1875453360430336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1415us (1761us total) trans 4294970544 rc 0/0 00000100:00100000:2.0:1788572544.521655:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4329 00000100:00100000:0.0:1788572544.525950:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.526404:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360430720 00000100:00100000:1.0:1788572544.526415:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360430720 00000100:00100000:1.0:1788572544.526499:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4330 00000100:00100000:1.0:1788572544.526511:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35927:x1875453360430720:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.526537:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.526540:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572544.526567:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163287 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdab expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.526585:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572544.526591:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.526604:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1632a3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdb2 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572544.527219:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x50:0x0], index 1: 0 00010000:00010000:1.0:1788572544.527425:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1632aa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.527439:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1632aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.527460:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1632aa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.527517:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1632a3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdb2 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.527532:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1632aa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdb2 expref: 89 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.527540:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627c5716100) 00010000:00010000:1.0:1788572544.527544:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572544.527547:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:1.0:1788572544.527603:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35927:x1875453360430720:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1095us (1664us total) trans 4294970545 rc 0/0 00000100:00100000:1.0:1788572544.527611:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4330 00000100:00100000:0.0:1788572544.529711:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.529753:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.529842:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360430848 00000100:00100000:2.0:1788572544.529862:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360430848 00000100:00100000:3.0:1788572544.529891:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360430976 00000100:00100000:3.0:1788572544.529895:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360430976 00000100:00100000:2.0:1788572544.529944:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4331 00000100:00100000:2.0:1788572544.529954:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35928:x1875453360430848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.529979:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.529999:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdb9 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530093:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530108:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530118:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530133:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530141:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530146:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530628:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1632cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530640:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1632cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530647:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1632cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.530673:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:2.0:1788572544.530677:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.530698:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1632cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530710:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530716:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530723:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530733:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00000100:00100000:3.0:1788572544.530734:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4332 00010000:00010000:2.0:1788572544.530737:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.530744:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35926:x1875453360430976:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.530745:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.530755:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.530757:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572544.530760:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.530763:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1632d4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdc0 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530765:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530772:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1632bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530784:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530790:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530797:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.530797:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530802:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.530805:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.530806:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530812:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.530814:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530824:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627e7a95f00) 00010000:00010000:2.0:1788572544.530827:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.530827:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530830:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.530834:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530836:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1632c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.530841:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530936:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdb9 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530951:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1632cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdb9 expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.530959:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572544.530961:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572544.530963:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572544.531099:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35928:x1875453360430848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1145us (1397us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.531107:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4331 00010000:00010000:3.0:1788572544.532311:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1632e9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532327:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1632e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532336:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1632e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.532364:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 16384 00000004:00200000:3.0:1788572544.532367:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.532383:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1632e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532393:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532400:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532410:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532416:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.532419:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532427:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532437:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572544.532440:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.532444:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532461:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb1632db lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532472:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532478:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532484:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532489:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.532492:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532503:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532511:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0xa01fa466 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.532514:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.532516:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532522:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1632e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0xa01fa466 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532554:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1632d4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc0 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532566:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1632e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc0 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.532573:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572544.532576:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572544.532579:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572544.532636:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35926:x1875453360430976:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1893us (2881us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.532645:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4332 00000100:00100000:0.0:1788572544.537235:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.537317:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360431232 00000100:00100000:2.0:1788572544.537328:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360431232 00000100:00100000:2.0:1788572544.537391:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4333 00000100:00100000:2.0:1788572544.537401:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35928:x1875453360431232:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.537475:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572544.537502:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163295 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efda4 expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.537527:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627e7a95a00) 00010000:00010000:2.0:1788572544.537545:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1632cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdb9 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.537555:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572544.537558:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572544.537581:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.537593:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 166 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.537600:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.537630:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 166 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.537642:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 166 pid: 18263 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.537657:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382368256:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.537737:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.541745:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.541797:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360431488 00000100:00100000:2.0:1788572544.541814:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360431488 00000100:00100000:2.0:1788572544.541875:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4334 00000100:00100000:2.0:1788572544.541885:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35927:x1875453360431488:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.541948:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572544.541976:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1632aa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdb2 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.541997:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627c5716100) 00010000:00010000:2.0:1788572544.542379:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1632e9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc0 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.542389:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572544.542394:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.542966:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382368256:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572544.542992:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.545302:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.545416:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360431616 00000100:00100000:2.0:1788572544.545427:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360431616 00010000:00010000:2.0:1788572544.545495:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 167 pid: 18263 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.545537:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1029 00000100:00100000:2.0:1788572544.545549:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec084050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:2347:x1875453360431616:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.545568:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.545576:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 167 pid: 18263 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.545590:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 167 pid: 18263 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.545615:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb15da66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede01 expref: 167 pid: 18263 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.545634:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572544.545639:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.545649:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.545656:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.545684:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.545746:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec084050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:2347:x1875453360431616:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 200us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.545755:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1029 00010000:00010000:0.0:1788572544.545869:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.545878:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.545902:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.545918:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.545926:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546269:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546278:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546406:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546413:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.546418:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546426:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546436:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572544.546439:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.546443:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546458:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1632f0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546470:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546476:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546481:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546484:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.546487:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546492:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546498:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x570b6fe0 (ffff9627e7451b00) 00010000:00010000:0.0:1788572544.546501:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.546504:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.546509:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1632f7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x570b6fe0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.546587:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35928:x1875453360431232:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 9186us (9667us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572544.546595:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4333 00010000:00010000:2.0:1788572544.546863:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.546874:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.546883:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.546897:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.546905:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.546912:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.547431:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.547440:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.547456:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548100:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.548106:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548115:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548129:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572544.548131:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.548137:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548143:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1632fe lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548155:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548161:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548168:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548174:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.548177:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548182:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548188:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x3fd6cf36 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572544.548190:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.548192:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.548197:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163305 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x3fd6cf36 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.548278:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35927:x1875453360431488:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 6392us (6540us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572544.550542:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4334 00000100:00100000:1.0:1788572544.552176:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.552338:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360431744 00000100:00100000:3.0:1788572544.552349:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360431744 00000100:00100000:3.0:1788572544.552410:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4335 00000100:00100000:3.0:1788572544.552420:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35927:x1875453360431744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.552445:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.552482:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16330c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdc7 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.552522:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.552536:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.552547:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.552650:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16330c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.552682:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.552692:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.552695:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.552700:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572544.552767:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35927:x1875453360431744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 349us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.552775:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4335 00000100:00100000:0.0:1788572544.553974:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.554995:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360431872 00000100:00100000:1.0:1788572544.555044:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360431872 00000100:00100000:1.0:1788572544.555120:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4336 00000100:00100000:1.0:1788572544.555954:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35928:x1875453360431872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.555976:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.556000:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16331a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdce expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.556085:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.556100:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.556115:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.556295:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb16331a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.556321:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 89 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.556332:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:0.0:1788572544.556335:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572544.556340:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:0.0:1788572544.556417:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35928:x1875453360431872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 477us (2448us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.558081:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.558203:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360432128 00000100:00100000:3.0:1788572544.558216:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360432128 00000100:00100000:1.0:1788572544.560769:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.561114:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360432384 00000100:00100000:2.0:1788572544.561128:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360432384 00000100:00100000:2.0:1788572544.561175:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4337 00000100:00100000:2.0:1788572544.561184:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35927:x1875453360432384:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.561356:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.561374:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.561387:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.562570:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4336 00000100:00100000:3.0:1788572544.563803:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4338 00000100:00100000:3.0:1788572544.563824:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35926:x1875453360432128:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.563892:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.564068:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163264 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efd9d expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564092:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627c5716900) 00010000:00010000:3.0:1788572544.564102:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572544.564126:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 3/0,1 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564141:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564152:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564168:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564176:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564183:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564483:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564491:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564496:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564502:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.564506:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564513:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564523:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572544.564525:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.564529:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 2/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564534:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb16332f lrc: 0/0,0 mode: --/CW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564544:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564549:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564555:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564559:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.564562:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564568:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564577:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xb16bf265 (ffff9627ffc51c00) 00010000:00010000:3.0:1788572544.564580:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.564582:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.564587:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163336 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0xb16bf265 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.565327:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35926:x1875453360432128:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1505us (7245us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572544.565339:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4338 00000040:00080000:2.0:1788572544.565774:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].65, 88 off13504 00000100:00100000:1.0:1788572544.566313:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.567139:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360432640 00000100:00100000:3.0:1788572544.567151:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360432640 00000100:00100000:3.0:1788572544.567218:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4339 00000100:00100000:3.0:1788572544.567229:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35928:x1875453360432640:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.567330:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.567349:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.567362:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572544.567642:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f21718ea reply state 0000000013fd7f2b transno 4294970546 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.567651:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572544.567752:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35927:x1875453360432384:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 6568us (6991us total) trans 4294970546 rc 0/0 00000100:00100000:2.0:1788572544.567761:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4337 00000040:00080000:3.0:1788572544.568044:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].68, 88 off13696 00000004:00010000:3.0:1788572544.568111:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000acad4537 reply state 000000002a42cdcb transno 4294970547 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.568119:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572544.568161:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35928:x1875453360432640:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 934us (1852us total) trans 4294970547 rc 0/0 00000100:00100000:3.0:1788572544.568169:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4339 00010000:00010000:2.0:1788572544.574401:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.574422:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.574429:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.574437:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.574442:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.574469:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.574487:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:2.0:1788572544.574490:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572544.574495:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.574503:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163328 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.575978:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572544.576138:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.576150:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.576151:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360432768 00010000:00010000:1.0:1788572544.576161:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.576161:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360432768 00010000:00010000:1.0:1788572544.576166:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.576170:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.576177:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.576189:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:1.0:1788572544.576192:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572544.576195:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.576209:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4340 00010000:00010000:1.0:1788572544.576213:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16333d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.576219:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35928:x1875453360432768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.576239:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.576256:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdd5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.576915:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.576932:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.576953:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:2.0:1788572544.576956:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572544.576959:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572544.577132:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35928:x1875453360432768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 913us (1160us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.577139:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4340 00000100:00100000:1.0:1788572544.578594:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.578681:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360432896 00000100:00100000:3.0:1788572544.578689:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360432896 00000100:00100000:3.0:1788572544.578739:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4341 00000100:00100000:3.0:1788572544.578749:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35926:x1875453360432896:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.578769:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.578791:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16334b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efddc expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.578826:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.578839:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.578850:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.578969:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb16334b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 93 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.578990:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 92 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.579000:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:3.0:1788572544.579183:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572544.579187:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd400, rc 0) 00000100:00100000:3.0:1788572544.579250:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35926:x1875453360432896:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 501us (662us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.579258:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4341 00000100:00100000:0.0:1788572544.582141:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.582178:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.582247:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360433152 00000100:00100000:2.0:1788572544.582259:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360433152 00000100:00100000:2.0:1788572544.582346:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1175 00000100:00100000:2.0:1788572544.582358:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360433152:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.582634:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360433024 00000100:00100000:3.0:1788572544.582640:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360433024 00000100:00100000:3.0:1788572544.582688:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4342 00000100:00100000:2.0:1788572544.582690:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360433152:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 336us (512us total) trans 4294967615 rc 0/0 00000100:00100000:3.0:1788572544.582696:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35927:x1875453360433024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.582698:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1175 00010000:00010000:3.0:1788572544.582711:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.582731:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efde3 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.582787:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.582808:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.582830:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.582834:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572544.582838:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc 0) 00000100:00100000:3.0:1788572544.582862:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35927:x1875453360433024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 167us (732us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.582869:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4342 00000100:00100000:0.0:1788572544.586987:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.587061:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360433280 00000100:00100000:3.0:1788572544.587072:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360433280 00000100:00100000:3.0:1788572544.587136:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1176 00000100:00100000:3.0:1788572544.587146:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360433280:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.587483:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360433280:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 337us (506us total) trans 4294967612 rc 0/0 00000100:00100000:3.0:1788572544.587491:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1176 00000100:00100000:1.0:1788572544.590151:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.590265:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360433536 00000100:00100000:0.0:1788572544.590274:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360433536 00000100:00100000:0.0:1788572544.590340:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4343 00000100:00100000:0.0:1788572544.590350:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35926:x1875453360433536:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.590444:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.590473:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.590487:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572544.590869:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].66, 88 off13592 00000004:00010000:0.0:1788572544.590972:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008e1ff8d1 reply state 00000000b56d53f4 transno 4294970548 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.590987:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572544.591131:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35926:x1875453360433536:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 781us (988us total) trans 4294970548 rc 0/0 00000100:00100000:0.0:1788572544.591140:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4343 00000100:00100000:0.0:1788572544.594292:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572544.594410:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.594425:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.594433:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.594440:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.594456:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.594467:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.594485:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:1.0:1788572544.594488:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572544.594493:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.594501:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163360 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.594908:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360433792 00000100:00100000:2.0:1788572544.594918:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360433792 00000100:00100000:2.0:1788572544.594980:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 920 00000100:00100000:2.0:1788572544.594992:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35928:x1875453360433792:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.595437:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35928:x1875453360433792:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 445us (1153us total) trans 4294970549 rc 0/0 00000100:00100000:2.0:1788572544.595456:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 920 00000100:00100000:3.0:1788572544.595538:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.596088:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360434048 00000100:00100000:2.0:1788572544.596095:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360434048 00000100:00100000:2.0:1788572544.596134:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 921 00000100:00100000:2.0:1788572544.596143:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35927:x1875453360434048:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.596378:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35927:x1875453360434048:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 236us (841us total) trans 4294970550 rc 0/0 00000100:00100000:2.0:1788572544.596386:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 921 00000100:00100000:3.0:1788572544.600225:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.600325:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360434176 00000100:00100000:0.0:1788572544.600336:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360434176 00000100:00100000:0.0:1788572544.600404:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4344 00000100:00100000:0.0:1788572544.600414:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35926:x1875453360434176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.600440:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.600472:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdea expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.600533:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.600550:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.600570:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:0.0:1788572544.600573:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.600577:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:0.0:1788572544.600645:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35926:x1875453360434176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 232us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.600654:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4344 00000100:00100000:0.0:1788572544.602475:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.602570:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360434432 00000100:00100000:3.0:1788572544.602579:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360434432 00000100:00100000:3.0:1788572544.602636:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4345 00000100:00100000:3.0:1788572544.602647:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35927:x1875453360434432:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.602827:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35927:x1875453360434432:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 180us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.602835:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4345 00000100:00100000:1.0:1788572544.606774:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.606818:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.607215:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360434816 00000100:00100000:0.0:1788572544.607228:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360434816 00000100:00100000:0.0:1788572544.607303:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4346 00000100:00100000:0.0:1788572544.607314:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35928:x1875453360434816:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.607436:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35928:x1875453360434816:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 124us (616us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.607454:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4346 00000100:00100000:3.0:1788572544.607591:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360434560 00000100:00100000:3.0:1788572544.607597:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360434560 00000100:00100000:3.0:1788572544.607633:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1177 00000100:00100000:3.0:1788572544.607640:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360434560:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.608146:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360434560:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 504us (1379us total) trans 4294967613 rc 0/0 00000100:00100000:3.0:1788572544.608155:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1177 00000100:00100000:0.0:1788572544.612476:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.613093:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.613169:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360435200 00000100:00100000:0.0:1788572544.613179:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360435200 00000100:00100000:0.0:1788572544.613248:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 922 00000100:00100000:0.0:1788572544.613259:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35926:x1875453360435200:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.613289:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360434944 00000100:00100000:2.0:1788572544.613294:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360434944 00000100:00100000:2.0:1788572544.613341:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4347 00000100:00100000:2.0:1788572544.613348:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35927:x1875453360434944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.613358:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.613379:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16336e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdf1 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.613444:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163375 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.613478:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163375 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.613489:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163375 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.613514:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.613532:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163375 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.613560:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16336e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf1 expref: 101 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.613576:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163375 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf1 expref: 100 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.613585:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:2.0:1788572544.613587:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572544.613591:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05400, rc 0) 00000100:00100000:2.0:1788572544.613651:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35927:x1875453360434944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 303us (1185us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.613658:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4347 00000100:00100000:0.0:1788572544.618092:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35926:x1875453360435200:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 4819us (4985us total) trans 4294970551 rc 0/0 00000100:00100000:0.0:1788572544.618109:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 922 00000100:00100000:1.0:1788572544.620995:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.621146:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.623048:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360435328 00000100:00100000:0.0:1788572544.623063:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360435328 00000100:00100000:0.0:1788572544.623117:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1178 00000100:00100000:0.0:1788572544.623128:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360435328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.623146:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.623167:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdf8 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572544.623195:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x50:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7380, blocks: 0x1 00010000:00010000:0.0:1788572544.623222:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=892s v=(0 0) 00010000:00010000:0.0:1788572544.623226:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.623238:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.623257:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.623272:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:0.0:1788572544.623326:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360435328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 200us (2341us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.623334:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1178 00000100:00100000:3.0:1788572544.625595:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360435456 00000100:00100000:3.0:1788572544.625609:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360435456 00000100:00100000:3.0:1788572544.625687:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4348 00000100:00100000:3.0:1788572544.625697:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35928:x1875453360435456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.625716:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.625736:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163383 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efdff expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.625809:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16338a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.625824:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16338a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.625836:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16338a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.626136:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.626160:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16338a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.626190:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163383 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdff expref: 101 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.626289:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16338a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdff expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.626300:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:3.0:1788572544.626303:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572544.626307:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572544.626362:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35928:x1875453360435456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 666us (5216us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.626370:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4348 00000100:00100000:0.0:1788572544.635965:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.636033:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.636063:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.636294:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360435584 00000100:00100000:1.0:1788572544.636305:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360435584 00000100:00100000:1.0:1788572544.636381:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1179 00000100:00100000:1.0:1788572544.636394:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360435584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.636414:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.636435:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe06 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572544.636474:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x50:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7380, blocks: 0x1 00010000:00010000:1.0:1788572544.636495:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=892s v=(0 0) 00010000:00010000:1.0:1788572544.636500:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572544.636511:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.636530:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.636544:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572544.636598:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360435584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 208us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.636605:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1179 00000100:00100000:3.0:1788572544.636764:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360435840 00000100:00100000:3.0:1788572544.636769:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360435840 00000100:00100000:3.0:1788572544.636800:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436096 00000100:00100000:3.0:1788572544.636803:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436096 00000100:00100000:3.0:1788572544.636821:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4349 00000100:00100000:3.0:1788572544.636829:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35926:x1875453360435840:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.636902:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35926:x1875453360435840:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 74us (897us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.636909:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4349 00000100:00100000:3.0:1788572544.636931:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4350 00000100:00100000:3.0:1788572544.636937:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35927:x1875453360436096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.636968:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35927:x1875453360436096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 31us (904us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.636976:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4350 00000100:00100000:0.0:1788572544.643128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.643169:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.643325:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436352 00000100:00100000:3.0:1788572544.643336:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436352 00000100:00100000:3.0:1788572544.643400:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1180 00000100:00100000:3.0:1788572544.643410:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35927:x1875453360436352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.643428:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.643458:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe14 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.643493:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.643502:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.643505:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.643512:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.643549:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.643564:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.643579:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382368384:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.643672:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.644439:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436224 00000100:00100000:1.0:1788572544.644459:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436224 00000100:00100000:1.0:1788572544.644526:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4351 00000100:00100000:1.0:1788572544.644536:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35926:x1875453360436224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.644549:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572544.644564:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16339f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe0d expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.644634:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb1633a6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.644649:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb1633a6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.644662:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb1633a6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572544.644700:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572544.644719:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb1633a6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.644746:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16339f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe0d expref: 102 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.644764:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb1633a6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe0d expref: 101 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.644774:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:1.0:1788572544.644777:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572544.644781:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97e00, rc 0) 00000100:00100000:1.0:1788572544.644826:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35926:x1875453360436224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 293us (1709us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.644833:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4351 00000100:00100000:0.0:1788572544.651583:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.651730:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382368384:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.651772:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.651795:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:1.0:1788572544.651866:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35927:x1875453360436352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8457us (8696us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.651876:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1180 00000100:00100000:3.0:1788572544.652526:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436608 00000100:00100000:3.0:1788572544.652537:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436608 00000100:00100000:3.0:1788572544.652589:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4352 00000100:00100000:3.0:1788572544.652599:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35928:x1875453360436608:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.652694:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35928:x1875453360436608:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 95us (1122us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.652702:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4352 00000100:00100000:0.0:1788572544.654247:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.654291:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.654371:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436736 00000100:00100000:3.0:1788572544.654382:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436736 00010000:00010000:3.0:1788572544.654420:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.654473:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1030 00000100:00100000:3.0:1788572544.654483:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360436736:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572544.654501:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.654528:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.654541:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.654567:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000bbaa91e/0x85ace67feb16337c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf8 expref: 14 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.654588:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.654592:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.654601:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.654612:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.654651:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.654669:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382368512:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572544.654753:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.654819:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436864 00000100:00100000:2.0:1788572544.654824:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436864 00000100:00100000:2.0:1788572544.654864:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1181 00000100:00100000:2.0:1788572544.654873:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360436864:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.654886:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.654903:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe1b expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572544.654921:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7380, blocks: 0x1 00010000:00010000:2.0:1788572544.654932:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=892s v=(0 0) 00010000:00010000:2.0:1788572544.654936:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572544.654943:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 15 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.654956:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 15 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.654967:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00000100:00100000:2.0:1788572544.654991:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360436864:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 120us (701us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.654998:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1181 00000100:00100000:3.0:1788572544.658243:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382368512:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572544.658270:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572544.658316:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360436736:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3834us (4077us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.658327:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1030 00000100:00100000:1.0:1788572544.663344:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.663397:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.663504:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360437248 00000100:00100000:0.0:1788572544.663515:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360437248 00000100:00100000:0.0:1788572544.663579:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4353 00000100:00100000:0.0:1788572544.663590:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35926:x1875453360437248:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.663705:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35926:x1875453360437248:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 116us (307us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.663714:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4353 00000100:00100000:2.0:1788572544.664165:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360436992 00000100:00100000:2.0:1788572544.664171:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360436992 00002000:00100000:2.0:1788572544.664216:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x50:0x0] (0->2) req@ffff9627fb30df80 x1875453360436992/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.664228:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30df80 with x1875453360436992 ext(0->2) 00002000:00010000:2.0:1788572544.664234:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30df80 00000100:00100000:2.0:1788572544.664260:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 263 00000100:00100000:2.0:1788572544.664268:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360436992:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.664710:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360436992:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 441us (1375us total) trans 4294967614 rc 0/0 00000100:00100000:2.0:1788572544.664719:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 263 00002000:00100000:2.0:1788572544.664726:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x50:0x0] (0->2) req@ffff9627fb30df80 x1875453360436992/t4294967614(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/448 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.664735:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30df80 with x1875453360436992 ext(0->2) 00002000:00010000:2.0:1788572544.664738:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30df80 00000100:00100000:1.0:1788572544.666396:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.666539:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360437376 00000100:00100000:0.0:1788572544.666549:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360437376 00000100:00100000:0.0:1788572544.666604:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1182 00000100:00100000:0.0:1788572544.666614:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35928:x1875453360437376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.666631:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.666650:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe22 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.666682:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.666692:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.666696:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.666701:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.666739:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.666754:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.666770:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f96d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382368640:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572544.666849:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.670643:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.670690:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.670799:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360437632 00000100:00100000:1.0:1788572544.670810:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360437632 00000100:00100000:1.0:1788572544.670887:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4354 00000100:00100000:1.0:1788572544.670899:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35927:x1875453360437632:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.671170:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360437504 00000100:00100000:3.0:1788572544.671176:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360437504 00000100:00100000:3.0:1788572544.671222:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1183 00000100:00100000:3.0:1788572544.671230:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35926:x1875453360437504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.671241:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.671258:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe29 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.671288:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 17 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.671298:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.671302:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.671308:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.671343:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 17 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.671356:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 17 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.671371:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382368768:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.671462:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.673691:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f96d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382368640:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572544.673729:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.673752:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094de00, rc 0) 00000100:00100000:2.0:1788572544.673802:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35928:x1875453360437376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 7190us (7413us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.673812:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1182 00000100:00100000:0.0:1788572544.683579:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.683607:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360437888 00000100:00100000:0.0:1788572544.683616:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360437888 00010000:00010000:0.0:1788572544.683657:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 18 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.683696:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1031 00000100:00100000:0.0:1788572544.683706:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360437888:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.683719:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.683744:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 18 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.683756:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 18 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.683778:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.683783:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.683792:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 18 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.683804:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 18 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.683839:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 18 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.684081:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb1633ad lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe1b expref: 18 pid: 13272 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.684098:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382368768:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.684125:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 17 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.684138:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc000, rc 0) 00000100:00100000:3.0:1788572544.684218:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35926:x1875453360437504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 12988us (13585us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.684228:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1183 00000100:00100000:0.0:1788572544.684774:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382368896:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572544.684832:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.684948:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.689798:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.689932:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382368896:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572544.689957:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.690038:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360437888:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6333us (6468us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.690050:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1031 00000100:00100000:0.0:1788572544.690079:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360437760 00000100:00100000:0.0:1788572544.690085:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360437760 00010000:00010000:0.0:1788572544.690116:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.690147:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1032 00000100:00100000:0.0:1788572544.690156:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360437760:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.690168:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.690190:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.690315:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.690341:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163391 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe06 expref: 14 pid: 6682 timeout: 992 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.690357:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.690363:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572544.690372:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.690384:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.690418:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.690433:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382369024:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572544.690496:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.693884:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382369024:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572544.693912:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.693972:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360437760:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3816us (9023us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.693985:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1032 00000100:00100000:2.0:1788572544.694933:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438016 00000100:00100000:2.0:1788572544.694944:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438016 00002000:00100000:2.0:1788572544.694986:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4f:0x0] (0->2) req@ffff9627fb30f480 x1875453360438016/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.694998:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30f480 with x1875453360438016 ext(0->2) 00002000:00010000:2.0:1788572544.695033:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30f480 00000100:00100000:2.0:1788572544.695064:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 264 00000100:00100000:2.0:1788572544.695074:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360438016:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.695503:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360438016:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 430us (5716us total) trans 4294967615 rc 0/0 00000100:00100000:2.0:1788572544.695511:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 264 00002000:00100000:2.0:1788572544.695519:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4f:0x0] (0->2) req@ffff9627fb30f480 x1875453360438016/t4294967615(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/448 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.695528:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30f480 with x1875453360438016 ext(0->2) 00002000:00010000:2.0:1788572544.695531:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30f480 00000100:00100000:2.0:1788572544.702245:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.702346:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.702370:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438272 00000100:00100000:2.0:1788572544.702380:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438272 00000100:00100000:2.0:1788572544.702427:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4355 00000100:00100000:2.0:1788572544.702437:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35926:x1875453360438272:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572544.702576:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35926:x1875453360438272:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 138us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.702584:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4355 00000100:00100000:0.0:1788572544.702949:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438144 00000100:00100000:0.0:1788572544.702955:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438144 00002000:00100000:0.0:1788572544.702988:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x50:0x0] (0->2) req@ffff9626c10a3100 x1875453360438144/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572544.703000:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a3100 with x1875453360438144 ext(0->2) 00002000:00010000:0.0:1788572544.703045:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a3100 00000100:00100000:0.0:1788572544.703062:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 265 00000100:00100000:0.0:1788572544.703069:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360438144:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.703471:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360438144:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 401us (1234us total) trans 4294967616 rc 0/0 00000100:00100000:0.0:1788572544.703478:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 265 00002000:00100000:0.0:1788572544.703485:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x50:0x0] (0->2) req@ffff9626c10a3100 x1875453360438144/t4294967616(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:205/0 lens 496/448 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572544.703494:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a3100 with x1875453360438144 ext(0->2) 00002000:00010000:0.0:1788572544.703497:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a3100 00000100:00100000:3.0:1788572544.708163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.708274:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438400 00000100:00100000:0.0:1788572544.708284:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438400 00000100:00100000:0.0:1788572544.708345:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4356 00000100:00100000:0.0:1788572544.708355:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35928:x1875453360438400:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.708496:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35928:x1875453360438400:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 141us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.708504:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4356 00000100:00100000:3.0:1788572544.708602:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.712092:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.712440:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438528 00000100:00100000:0.0:1788572544.712461:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438528 00000100:00100000:0.0:1788572544.712524:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438656 00000100:00100000:0.0:1788572544.712527:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438656 00000100:00100000:0.0:1788572544.712552:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1184 00000100:00100000:0.0:1788572544.712563:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2347:x1875453360438528:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.712887:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1185 00000100:00100000:0.0:1788572544.712894:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360438656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572544.756954:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35927:x1875453360437632:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 86048us (86252us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.756969:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4354 00000001:00080000:2.0:1788572544.758799:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970551 is committed 00000020:00080000:2.0:1788572544.758842:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 120 batchid updated 00000020:00080000:2.0:1788572544.758899:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ff5483c0 refcount 2 committed 0 result 0 batchid 121 00000020:00080000:2.0:1788572544.758905:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3b00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d188ac00 00000020:00080000:2.0:1788572544.758909:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4037000 00000020:00080000:2.0:1788572544.758989:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2f643c0 refcount 2 committed 0 result 0 batchid 123 00000020:00080000:2.0:1788572544.758993:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31def00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9628008d8900 00000020:00080000:2.0:1788572544.758996:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f5b01000 00000020:00080000:2.0:1788572544.759030:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c8550e40 refcount 2 committed 0 result 0 batchid 122 00000020:00080000:2.0:1788572544.759034:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c9bda300 00000020:00080000:2.0:1788572544.759037:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9626c31de000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f5980000 00000020:00080000:3.0:1788572544.759750:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 123 committed batchid 120 00000020:00080000:3.0:1788572544.762444:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 120 00000100:00100000:2.0:1788572544.762634:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.763367:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360438784 00000100:00100000:0.0:1788572544.763379:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360438784 00000100:00100000:0.0:1788572544.763485:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1186 00000100:00100000:0.0:1788572544.763497:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360438784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572544.763616:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360438784:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 121us (987us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.763624:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1186 00000001:00080000:1.0:1788572544.768570:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967610 is committed 00000001:00080000:1.0:1788572544.768589:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967615 is committed 00000001:00080000:1.0:1788572544.768607:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967611 is committed 00000100:00100000:0.0:1788572544.769113:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360438528:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 56546us (60504us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.769125:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1184 00000100:00100000:3.0:1788572544.773470:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.776818:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360439040 00000100:00100000:1.0:1788572544.776833:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360439040 00000100:00100000:1.0:1788572544.776911:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4357 00000100:00100000:1.0:1788572544.776931:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35927:x1875453360439040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.776982:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.777028:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163375 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdf1 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.777047:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:1.0:1788572544.777052:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572544.777083:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.777096:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.777107:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572544.777478:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006c7d998d reply state 0000000036c9eedc transno 4294970552 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.777495:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572544.777650:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35927:x1875453360439040:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 721us (4200us total) trans 4294970552 rc 0/0 00000100:00100000:1.0:1788572544.777659:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4357 00000001:00080000:2.0:1788572544.779020:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967616 is committed 00000001:00080000:2.0:1788572544.779040:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967613 is committed 00000001:00080000:2.0:1788572544.779052:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967614 is committed 00000100:00100000:0.0:1788572544.779807:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360438656:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 66905us (67718us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.779820:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1185 00000100:00100000:3.0:1788572544.779995:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572544.784056:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.784075:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.784083:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.784089:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.784095:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.784104:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.784123:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:1.0:1788572544.784126:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.784131:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.784138:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1633c2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.784443:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.784489:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.784604:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360439424 00000100:00100000:1.0:1788572544.784614:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360439424 00000100:00100000:1.0:1788572544.784684:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1187 00000100:00100000:1.0:1788572544.784695:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360439424:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572544.784981:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x50:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572544.784985:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x50:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:1.0:1788572544.784987:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x50:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:1.0:1788572544.784990:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x50:0x0] updating lvb ctime from disk: 1788572544 -> 1788572527 00002000:00010000:1.0:1788572544.784992:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x50:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572544.785070:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360439424:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 376us (629us total) trans 4294967616 rc 0/0 00000100:00100000:1.0:1788572544.785078:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1187 00000100:00100000:0.0:1788572544.792020:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360439296 00000100:00100000:0.0:1788572544.792038:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360439296 00000100:00100000:0.0:1788572544.792111:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360439680 00000100:00100000:0.0:1788572544.792115:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360439680 00000100:00100000:0.0:1788572544.792145:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4358 00000100:00100000:0.0:1788572544.792155:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35926:x1875453360439296:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.792228:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.792257:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97e00/0x85ace67feb1633a6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe0d expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.792280:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:0.0:1788572544.792285:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572544.792316:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.792331:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.792342:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572544.792528:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000004e472d reply state 000000003e91e0f7 transno 4294970553 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.792539:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572544.792650:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35926:x1875453360439296:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 495us (12656us total) trans 4294970553 rc 0/0 00000100:00100000:0.0:1788572544.792658:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4358 00000100:00100000:0.0:1788572544.792680:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4359 00000100:00100000:0.0:1788572544.792687:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35928:x1875453360439680:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.792710:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.792724:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16338a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdff expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.792737:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:0.0:1788572544.792741:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572544.792755:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.792764:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.792771:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572544.792842:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000095af120f reply state 000000005c1e7099 transno 4294970554 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.792850:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572544.792885:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35928:x1875453360439680:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 198us (8395us total) trans 4294970554 rc 0/0 00000100:00100000:0.0:1788572544.792891:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4359 00000100:00100000:3.0:1788572544.796421:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.796702:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360439936 00000100:00100000:0.0:1788572544.796713:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360439936 00000100:00100000:0.0:1788572544.796771:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4360 00000100:00100000:0.0:1788572544.796781:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35927:x1875453360439936:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.796847:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.796866:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.796877:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.796926:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.796941:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.796957:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382369152:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572544.797063:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b0b40 going to sleep for 11 seconds 00010000:00010000:1.0:1788572544.797530:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.797540:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.797548:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.797554:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.797559:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.797567:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.797582:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:1.0:1788572544.797585:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.797590:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.797597:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1633c9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.799038:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.799138:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360440064 00000100:00100000:3.0:1788572544.799147:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360440064 00000100:00100000:3.0:1788572544.799209:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1188 00000100:00100000:3.0:1788572544.799220:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360440064:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572544.799503:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x50:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572544.799507:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x50:0x0] updating lvb mtime from disk: 0 -> 1788572522 00002000:00010000:3.0:1788572544.799510:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x50:0x0] updating lvb atime from disk: 0 -> 1788572522 00002000:00010000:3.0:1788572544.799512:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x50:0x0] updating lvb ctime from disk: 1788572544 -> 1788572523 00002000:00010000:3.0:1788572544.799515:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x50:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572544.799578:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360440064:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 359us (582us total) trans 4294967617 rc 0/0 00000100:00100000:3.0:1788572544.799585:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1188 00010000:00010000:1.0:1788572544.799929:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.799941:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.799950:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.799957:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572544.799962:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.799971:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.799989:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:1.0:1788572544.799993:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572544.799998:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.800041:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1633d0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.803099:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.803309:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360440192 00000100:00100000:2.0:1788572544.803320:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360440192 00000100:00100000:2.0:1788572544.803387:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1189 00000100:00100000:2.0:1788572544.803398:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7661c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360440192:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572544.803665:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572544.803670:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb mtime from disk: 0 -> 1788572516 00002000:00010000:2.0:1788572544.803673:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb atime from disk: 0 -> 1788572516 00002000:00010000:2.0:1788572544.803676:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb ctime from disk: 1788572544 -> 1788572527 00002000:00010000:2.0:1788572544.803678:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572544.803751:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7661c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360440192:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 354us (663us total) trans 4294967617 rc 0/0 00000100:00100000:2.0:1788572544.803760:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1189 00000100:00100000:3.0:1788572544.804433:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382369152:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.804471:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.807935:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.807995:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.808216:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360440576 00000100:00100000:2.0:1788572544.808228:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360440576 00000100:00100000:2.0:1788572544.808295:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4361 00000100:00100000:2.0:1788572544.808305:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35928:x1875453360440576:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.808385:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.808403:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.808414:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 4/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.808481:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.808494:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 66 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.808508:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382369280:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.808594:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.809288:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360440320 00000100:00100000:0.0:1788572544.809295:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360440320 00010000:00010000:0.0:1788572544.809325:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.809362:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1033 00000100:00100000:0.0:1788572544.809370:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360440320:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.809381:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.809387:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.809399:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.809417:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163313 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x92:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdc7 expref: 67 pid: 9040 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.809432:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:0.0:1788572544.809437:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.809456:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.809464:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.809491:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.809529:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360440320:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 160us (1616us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.809536:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1033 00010000:00010000:3.0:1788572544.809935:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.809948:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.809982:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.809990:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.809997:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.810039:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.810045:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.810054:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.810067:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:3.0:1788572544.810070:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572544.810076:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.810082:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1633d7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.810148:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35927:x1875453360439936:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13368us (13736us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572544.810159:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4360 00000100:00100000:1.0:1788572544.813938:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.814110:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360440832 00000100:00100000:3.0:1788572544.814119:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360440832 00000100:00100000:3.0:1788572544.814180:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4362 00000100:00100000:3.0:1788572544.814190:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35926:x1875453360440832:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.814272:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.814289:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.814305:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.814340:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.814353:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 65 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.814369:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382369408:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.814461:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b79a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.815239:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382369280:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572544.815264:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.817612:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.817688:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360440960 00000100:00100000:1.0:1788572544.817698:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360440960 00000100:00100000:1.0:1788572544.817769:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1190 00000100:00100000:1.0:1788572544.817782:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360440960:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572544.817824:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/1,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.817846:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.817856:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572544.817861:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572544.817866:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 4/1,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.817903:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 13 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.817927:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 13 pid: 20911 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.817943:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382369536:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.818095:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.823679:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.823750:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.823898:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382369408:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.823933:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.824053:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441088 00000100:00100000:2.0:1788572544.824061:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441088 00010000:00010000:2.0:1788572544.824101:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 66 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.824138:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441216 00000100:00100000:2.0:1788572544.824141:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441216 00010000:00010000:2.0:1788572544.824153:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 67 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.824177:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1034 00000100:00100000:2.0:1788572544.824185:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360441088:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.824210:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.824218:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 67 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824230:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 67 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824253:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163321 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x91:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdce expref: 67 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824269:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:2.0:1788572544.824274:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824283:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824291:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824316:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.824360:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360441088:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 176us (692us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.824368:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1034 00000100:00100000:2.0:1788572544.824388:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1035 00000100:00100000:2.0:1788572544.824394:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360441216:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.824403:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.824408:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 65 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824416:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 65 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824427:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163352 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x90:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efddc expref: 65 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824437:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:2.0:1788572544.824442:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824460:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824467:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824480:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.824498:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360441216:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 103us (748us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.824504:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1035 00010000:00010000:2.0:1788572544.824522:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824533:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824560:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824568:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824574:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824579:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.824583:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824591:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824599:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:2.0:1788572544.824602:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572544.824606:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 2/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.824612:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1633de lrc: 0/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.824655:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35928:x1875453360440576:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 16351us (16659us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572544.824663:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4361 00010000:00010000:3.0:1788572544.824881:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824889:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824905:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824923:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824930:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824936:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.824939:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824946:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824956:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:3.0:1788572544.824959:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572544.824962:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.824968:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1633e5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.825001:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35926:x1875453360440832:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10811us (11083us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572544.825060:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4362 00000100:00100000:1.0:1788572544.825261:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382369536:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.825297:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/1,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.829189:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.829301:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.829376:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441472 00000100:00100000:3.0:1788572544.829387:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441472 00000100:00100000:3.0:1788572544.829453:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1191 00000100:00100000:3.0:1788572544.829465:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360441472:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.829511:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.829532:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.829547:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.829552:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572544.829556:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.829591:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.829604:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 11 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.829619:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382369664:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.829695:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b79a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.830357:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441344 00000100:00100000:1.0:1788572544.830364:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441344 00010000:00010000:1.0:1788572544.830404:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 14 pid: 20911 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.830438:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1036 00000100:00100000:1.0:1788572544.830457:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec087c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360441344:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572544.830470:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.830492:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 14 pid: 20911 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.830506:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 14 pid: 20911 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.830526:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb163398 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe14 expref: 14 pid: 20911 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.830540:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572544.830543:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572544.830549:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.830558:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.830595:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572544.830638:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec087c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360441344:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 192us (1459us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.830649:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1036 00010000:00010000:3.0:1788572544.830754:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830764:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830787:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830795:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830807:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830814:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.830818:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830826:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830838:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 2/0,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.830857:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb1633ec lrc: 0/0,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.830901:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360440960:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 13122us (13298us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.830908:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1190 00000100:00100000:0.0:1788572544.833565:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.833654:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441600 00000100:00100000:2.0:1788572544.833664:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441600 00000100:00100000:2.0:1788572544.833719:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1192 00000100:00100000:2.0:1788572544.833729:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360441600:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.833775:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.833795:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 12 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.833804:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572544.833808:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572544.833813:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 4/1,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.833865:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 12 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.833878:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 12 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.833893:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382369792:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.833985:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.838297:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.838473:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382369664:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.838504:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.838581:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382369792:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.838601:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/1,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.838664:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441856 00000100:00100000:2.0:1788572544.838671:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441856 00000100:00100000:2.0:1788572544.838728:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4363 00000100:00100000:2.0:1788572544.838738:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35927:x1875453360441856:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.838822:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.838837:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.838848:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.839136:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.839149:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 63 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.839160:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.839199:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 63 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.839213:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 63 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.839231:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382369920:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572544.839319:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7300 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.843118:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.843157:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.843424:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360442112 00000100:00100000:0.0:1788572544.843433:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360442112 00010000:00010000:0.0:1788572544.843486:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.843529:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360441984 00000100:00100000:0.0:1788572544.843533:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360441984 00010000:00010000:0.0:1788572544.843548:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 12 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.843575:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1037 00000100:00100000:0.0:1788572544.843584:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360442112:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.843600:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.843624:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843638:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843662:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000095d50bab/0x85ace67feb1633bb lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe29 expref: 13 pid: 6681 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843681:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.843685:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.843693:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843704:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843733:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.843781:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360442112:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 198us (674us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.843789:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1037 00000100:00100000:0.0:1788572544.843808:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1038 00000100:00100000:0.0:1788572544.843815:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360441984:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.843823:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.843837:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 12 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843862:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 12 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843877:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb1633b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x50:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe22 expref: 12 pid: 6682 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843891:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.843894:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572544.843899:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843907:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843935:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.843953:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360441984:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 138us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.843960:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1038 00010000:00010000:0.0:1788572544.843984:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.843994:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844055:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844065:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844073:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844082:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.844087:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844096:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844108:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844117:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ab22e0e/0x85ace67feb1633f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.844173:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360441472:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 14709us (14871us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.844181:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1191 00010000:00010000:0.0:1788572544.844371:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844380:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844398:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844411:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844419:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844425:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.844428:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844436:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844458:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.844466:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1633fa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.844507:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360441600:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 10779us (10951us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.844514:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1192 00000100:00100000:2.0:1788572544.845145:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131d180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382369920:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572544.845170:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.848308:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.849666:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.850027:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360442496 00000100:00100000:3.0:1788572544.850038:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360442496 00000100:00100000:3.0:1788572544.850099:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4364 00000100:00100000:3.0:1788572544.850110:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35926:x1875453360442496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.850192:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850224:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850238:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850484:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850497:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850506:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850543:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.850557:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 65 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.850573:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382370048:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.850660:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b79a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.851831:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360442240 00000100:00100000:2.0:1788572544.851840:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360442240 00010000:00010000:2.0:1788572544.851895:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 66 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.851943:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1039 00000100:00100000:2.0:1788572544.851953:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360442240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.851967:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.851973:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 66 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.851988:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 66 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.852025:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163359 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efde3 expref: 66 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.852041:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:2.0:1788572544.852046:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.852056:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.852064:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.852095:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.852143:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360442240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 192us (3847us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.852152:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1039 00010000:00010000:3.0:1788572544.852428:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.852438:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572544.852629:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970555 00000001:00080000:3.0:1788572544.852787:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970557 00000001:00080000:3.0:1788572544.852830:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970558 00000100:00100000:3.0:1788572544.852941:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcd500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382370176:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572544.852995:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572544.854611:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.855990:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382370176 00000100:00100000:2.0:1788572544.856025:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382370176 00000100:00100000:2.0:1788572544.856090:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1788572544.856100:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382370176:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572544.856357:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382370176:12345-0@lo:1000:mdt00_000.0 Request processed in 258us (3361us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.856366:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1788572544.856443:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967611 gen 2 00000004:00080000:0.0:1788572544.856461:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967611 committed transno 4294967611 00000100:00100000:0.0:1788572544.856486:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcd500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382370176:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1788572544.856898:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970559 00000100:00100000:0.0:1788572544.856995:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbe680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382370304:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572544.857072:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572544.857100:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1151600 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.857701:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382370304 00000100:00100000:3.0:1788572544.857709:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382370304 00000100:00100000:3.0:1788572544.857758:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:3.0:1788572544.857767:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382370304:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572544.857992:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382370304:12345-0@lo:1000:mdt00_000.0 Request processed in 225us (919us total) trans 4294967618 rc 0/0 00000100:00100000:3.0:1788572544.857999:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:0.0:1788572544.858334:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbe680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382370304:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1788572544.858375:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858385:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858393:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858398:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.858402:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858410:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858428:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627ffc51700) 00010000:00010000:0.0:1788572544.858431:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.858437:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858443:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163408 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x92:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858487:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858494:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858499:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858503:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.858506:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858511:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858518:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:0.0:1788572544.858520:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.858523:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858529:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb15f7fc lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858552:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858565:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858575:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858581:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858588:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858593:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858598:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.858601:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858607:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858615:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:0.0:1788572544.858617:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.858620:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858626:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb16341d lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858638:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858644:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858649:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858653:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.858656:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858662:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858669:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:0.0:1788572544.858671:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572544.858674:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.858679:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb163401 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.858998:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35927:x1875453360441856:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 20107us (20558us total) trans 4294970560 rc 0/0 00000100:00100000:0.0:1788572544.859088:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4363 00000020:00080000:0.0:1788572544.859093:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:2.0:1788572544.860376:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.860593:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.860651:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360442880 00000100:00100000:0.0:1788572544.860660:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360442880 00010000:00010000:0.0:1788572544.860697:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 68 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.860738:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1040 00000100:00100000:0.0:1788572544.860747:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360442880:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572544.860761:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572544.860768:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 68 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.860783:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 68 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.860805:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:0.0:1788572544.860811:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.860822:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.860831:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.860868:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572544.860912:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360442880:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 166us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.860925:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1040 00010000:00010000:0.0:1788572544.860970:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163367 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdea expref: 67 pid: 5838 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.860989:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382370048:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572544.861032:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.861043:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.861049:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572544.861239:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970556 00000001:00080000:0.0:1788572544.861396:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970561 00000001:00080000:0.0:1788572544.861459:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970562 00000100:00100000:0.0:1788572544.861548:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382370432:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572544.861598:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572544.861627:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.861647:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382370432 00000100:00100000:0.0:1788572544.861652:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382370432 00000100:00100000:0.0:1788572544.861682:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1788572544.861690:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382370432:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572544.861751:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382370432:12345-0@lo:1000:mdt00_002.0 Request processed in 62us (153us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.861759:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1788572544.861822:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382370432:0@lo:1000:mdt00_002.0 00000001:00080000:0.0:1788572544.862172:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970563 00000100:00100000:0.0:1788572544.862217:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382370560:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572544.862245:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572544.862261:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b79a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.862274:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382370560 00000100:00100000:0.0:1788572544.862278:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382370560 00000100:00100000:0.0:1788572544.862301:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1788572544.862307:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382370560:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572544.862465:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:5839:x1875453382370560:12345-0@lo:1000:mdt00_002.0 Request processed in 157us (219us total) trans 4294967619 rc 0/0 00000100:00100000:0.0:1788572544.862472:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1788572544.862507:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382370560:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1788572544.862534:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862548:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862555:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862562:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.862567:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862575:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862589:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627ffc51c00) 00010000:00010000:0.0:1788572544.862592:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.862597:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862603:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cca00/0x85ace67feb163416 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x90:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862633:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862640:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862646:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862651:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23eee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.862654:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862661:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862670:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:0.0:1788572544.862673:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572544.862681:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 165 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862693:0:5839:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 165 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862703:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 165 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.862728:0:5839:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53400/0x85ace67feb16257c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef991 expref: 165 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.862740:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382370688:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572544.862796:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.870175:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.870294:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.870543:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360442752 00000100:00100000:3.0:1788572544.870554:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360442752 00000100:00100000:3.0:1788572544.870613:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4365 00000100:00100000:3.0:1788572544.870623:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35928:x1875453360442752:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572544.870707:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.870725:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.870738:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.871000:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.871032:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 72 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.871042:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 4/0,1 mode: --/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.871077:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 72 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.871093:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 72 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572544.871109:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382370816:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572544.871187:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1317c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572544.871637:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382370688:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572544.871663:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.871671:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb15f7c4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.871695:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.871705:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 165 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.871712:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.871737:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 165 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.871746:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 165 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.871757:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382370944:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572544.871796:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0776c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.873587:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360443136 00000100:00100000:2.0:1788572544.873599:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360443136 00000100:00100000:2.0:1788572544.873664:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360443264 00000100:00100000:2.0:1788572544.873667:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360443264 00000100:00100000:2.0:1788572544.873689:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 923 00000100:00100000:2.0:1788572544.873698:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35927:x1875453360443136:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572544.874414:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x92:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572544.874536:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970566 00010000:00010000:2.0:1788572544.874587:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.874608:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.874621:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.874635:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.874642:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.874648:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.874677:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572544.874783:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:35927:x1875453360443136:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1084us (4615us total) trans 4294970567 rc 0/0 00000100:00100000:2.0:1788572544.874793:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 923 00000020:00080000:2.0:1788572544.874798:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788572544.874820:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 924 00000100:00100000:2.0:1788572544.874830:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360443264:12345-192.168.202.40@tcp:49:diff.0 00000100:00100000:2.0:1788572544.874899:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360443264:12345-192.168.202.40@tcp:49:diff.0 Request processed in 71us (4605us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.874908:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 924 00010000:00010000:3.0:1788572544.875136:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572544.875145:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.875152:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.875157:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.875165:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.875177:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x92:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:3.0:1788572544.875181:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572544.875185:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.875201:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163439 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x92:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.875219:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.875220:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.875222:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572544.878644:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382370816:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572544.878679:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.881498:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.881653:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360443392 00000100:00100000:3.0:1788572544.881664:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360443392 00000100:00100000:3.0:1788572544.881740:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1193 00000100:00100000:3.0:1788572544.881752:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:35867:x1875453360443392:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572544.881771:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.881795:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b97b34a8/0x85ace67feb163440 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe30 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572544.881823:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4f:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572544.881844:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=892s v=(0 0) 00010000:00010000:3.0:1788572544.881850:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.881861:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b97b34a8/0x85ace67feb163440 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe30 expref: 48 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.881878:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b97b34a8/0x85ace67feb163440 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe30 expref: 48 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.881891:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cca00, rc 0) 00000100:00100000:3.0:1788572544.881947:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:35867:x1875453360443392:12345-192.168.202.40@tcp:101:diff.0 Request processed in 198us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.881955:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1193 00000100:00100000:1.0:1788572544.882130:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382370944:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572544.882169:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788572544.883596:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788572544.885946:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.885988:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.886595:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360443648 00000100:00100000:1.0:1788572544.886606:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360443648 00010000:00010000:1.0:1788572544.886665:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 166 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572544.886707:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360443520 00000100:00100000:1.0:1788572544.886711:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360443520 00010000:00010000:1.0:1788572544.886729:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 75 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.886907:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1041 00000100:00100000:2.0:1788572544.886932:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:2349:x1875453360443648:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572544.886950:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572544.886958:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 166 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.886977:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 166 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.887029:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb162521 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ef975 expref: 166 pid: 23058 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.887049:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:2.0:1788572544.887055:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.887066:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.887074:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.887103:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572544.887156:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:2349:x1875453360443648:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 238us (1219us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.887822:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1042 00000100:00100000:1.0:1788572544.887840:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360443520:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572544.887865:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572544.887874:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 75 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.887887:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 75 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.887902:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163344 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efdd5 expref: 75 pid: 5839 timeout: 992 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.889763:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:1.0:1788572544.889778:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.889792:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.889801:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572544.890333:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572544.890398:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360443520:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 2563us (4404us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572544.890410:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1042 00000100:00100000:1.0:1788572544.890515:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.891637:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1041 00010000:00010000:2.0:1788572544.891685:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891696:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891713:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891725:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891733:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891739:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.891744:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891752:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891770:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:2.0:1788572544.891774:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.891779:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891786:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163432 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891800:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891806:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891811:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891816:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.891819:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891825:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.891833:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffff9627e7a95800) 00010000:00010000:2.0:1788572544.891836:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572544.891838:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.892155:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16340f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.892486:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35926:x1875453360442496:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 42374us (42823us total) trans 4294970564 rc 0/0 00000100:00100000:2.0:1788572544.892496:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4364 00000020:00080000:2.0:1788572544.892500:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572544.892609:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360443776 00000100:00100000:3.0:1788572544.892618:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360443776 00002000:00100000:3.0:1788572544.892659:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4f:0x0] (0->4095) req@ffff9627fb30ed80 x1875453360443776/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:205/0 lens 488/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572544.892670:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30ed80 with x1875453360443776 ext(0->4095) 00002000:00010000:3.0:1788572544.892674:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30ed80 00000100:00100000:3.0:1788572544.892697:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 266 00000100:00100000:3.0:1788572544.892707:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2347:x1875453360443776:12345-192.168.202.40@tcp:3:diff.0 00010000:00010000:2.0:1788572544.893227:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.893237:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572544.893391:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970565 00000001:00080000:2.0:1788572544.893557:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970568 00000001:00080000:2.0:1788572544.893606:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970569 00000100:00100000:2.0:1788572544.893693:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382371072:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1788572544.893756:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.893798:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb0db240 going to sleep for 11 seconds 00000100:00100000:0.0:1788572544.893974:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382371072 00000100:00100000:0.0:1788572544.893981:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382371072 00000100:00100000:0.0:1788572544.894043:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1788572544.894052:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382371072:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572544.894119:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382371072:12345-0@lo:1000:mdt00_001.0 Request processed in 68us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.894127:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:2.0:1788572544.894354:0:5838:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967614 gen 2 00000004:00080000:2.0:1788572544.894360:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967613 committed transno 4294967614 00000004:00080000:2.0:1788572544.894379:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967614 committed transno 4294967614 00000100:00100000:2.0:1788572544.894393:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382371072:0@lo:1000:mdt00_001.0 00000001:00080000:2.0:1788572544.894730:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970570 00000100:00100000:2.0:1788572544.894785:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382371200:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1788572544.894817:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572544.894838:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572544.894991:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2347:x1875453360443776:12345-192.168.202.40@tcp:3:diff.0 Request processed in 2281us (4471us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572544.895001:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 266 00002000:00100000:3.0:1788572544.895029:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4f:0x0] (0->4095) req@ffff9627fb30ed80 x1875453360443776/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:205/0 lens 488/4536 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572544.895038:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30ed80 with x1875453360443776 ext(0->4095) 00002000:00010000:3.0:1788572544.895043:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30ed80 00000100:00100000:2.0:1788572544.897585:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382371200 00000100:00100000:2.0:1788572544.897604:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382371200 00000100:00100000:2.0:1788572544.897680:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:2.0:1788572544.897691:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382371200:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1788572544.898047:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382371200:12345-0@lo:1000:mdt00_001.0 Request processed in 357us (3227us total) trans 4294967618 rc 0/0 00000100:00100000:2.0:1788572544.898056:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:2.0:1788572544.898124:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382371200:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1788572544.898174:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898185:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898194:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898325:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.898332:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898342:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898364:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627c5716400) 00010000:00010000:2.0:1788572544.898368:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572544.898375:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898382:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16342b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x91:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898417:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898423:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898430:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898435:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.898438:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898456:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898464:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572544.898467:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572544.898470:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898476:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb15f8a4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898505:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898517:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898528:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898535:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898542:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898547:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898553:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.898555:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898561:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898569:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572544.898571:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.898574:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898578:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163447 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898589:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898595:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898601:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898606:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.898609:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898615:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898622:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572544.898625:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572544.898627:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.898634:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163424 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.898813:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35928:x1875453360442752:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 28188us (38441us total) trans 4294970571 rc 0/0 00000100:00100000:2.0:1788572544.898825:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4365 00000020:00080000:2.0:1788572544.898829:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:2.0:1788572544.899831:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572544.899838:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572544.899872:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 151 count 101 00000040:00080000:2.0:1788572544.899879:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:2.0:1788572544.900095:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7848000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382371328:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572544.900165:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d188ac00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572544.900753:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.901042:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360444032 00000100:00100000:1.0:1788572544.901051:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360444032 00000100:00100000:1.0:1788572544.901122:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 925 00000100:00100000:1.0:1788572544.901134:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35926:x1875453360444032:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572544.901713:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x90:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572544.901754:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970572 00010000:00010000:2.0:1788572544.901796:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.901812:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.901824:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.901832:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.901839:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.901858:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.901928:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572544.902035:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35926:x1875453360444032:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 902us (1284us total) trans 4294970573 rc 0/0 00000100:00100000:2.0:1788572544.902044:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 925 00000020:00080000:2.0:1788572544.902049:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572544.906156:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572544.906173:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.906184:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572544.906189:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.906212:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.906233:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x90:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:0.0:1788572544.906236:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572544.906241:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 1/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.906248:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16344e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x90:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.906276:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.906278:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.906280:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572544.906689:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572544.906770:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360444288 00000100:00100000:1.0:1788572544.906786:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360444288 00000100:00100000:0.0:1788572544.909864:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 926 00000100:00100000:0.0:1788572544.909891:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+165:35867:x1875453360444288:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572544.915186:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360444288:12345-192.168.202.40@tcp:35:diff.0 Request processed in 5292us (8486us total) trans 4294970574 rc 0/0 00000100:00100000:0.0:1788572544.915212:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 926 00000100:00100000:1.0:1788572544.915419:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.916257:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360444544 00000100:00100000:0.0:1788572544.916270:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360444544 00000100:00100000:0.0:1788572544.916324:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 927 00000100:00100000:0.0:1788572544.916333:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35928:x1875453360444544:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572544.916708:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x91:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572544.916750:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970575 00010000:00010000:0.0:1788572544.916799:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.916820:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.916834:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.916843:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.916849:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.916856:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.916886:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572544.916979:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35928:x1875453360444544:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 645us (1560us total) trans 4294970576 rc 0/0 00000100:00100000:0.0:1788572544.916991:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 927 00000020:00080000:0.0:1788572544.916997:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572544.917291:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572544.917300:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.917309:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572544.917315:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.917324:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.917337:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x91:0x0].0x0 (ffff9627e7451500) 00010000:00010000:3.0:1788572544.917341:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572544.917347:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.917354:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163455 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x91:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.917380:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.917382:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572544.917384:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572544.921917:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.922053:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360444672 00000100:00100000:3.0:1788572544.922066:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360444672 00000100:00100000:3.0:1788572544.922121:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4366 00000100:00100000:3.0:1788572544.922130:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+166:35867:x1875453360444672:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572544.922158:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.922181:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16345c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe37 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922265:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922279:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922292:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922308:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922316:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922323:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922621:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163471 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922630:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163471 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922637:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163471 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572544.922671:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xae:0x0]: returning size 3 00000004:00200000:3.0:1788572544.922676:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572544.922696:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163471 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922709:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163471 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922718:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922725:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922736:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922742:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.922747:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922756:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922768:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572544.922772:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.922777:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922785:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc000/0x85ace67feb163463 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922798:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922804:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922810:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922815:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572544.922818:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922828:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922835:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x6e9b5ca3 (ffff9627ffc51d00) 00010000:00010000:3.0:1788572544.922837:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572544.922840:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922846:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd600/0x85ace67feb16346a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922877:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16345c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe37 expref: 168 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922890:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd400/0x85ace67feb163471 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe37 expref: 167 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.922896:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:3.0:1788572544.922899:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572544.922901:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd400, rc 0) 00000100:00100000:3.0:1788572544.922966:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360444672:12345-192.168.202.40@tcp:101:diff.0 Request processed in 837us (1066us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.922975:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4366 00000100:00100000:3.0:1788572544.927515:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7848000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382371328:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572544.930934:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.931054:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360444800 00000100:00100000:0.0:1788572544.931065:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360444800 00000100:00100000:0.0:1788572544.931120:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4367 00000100:00100000:0.0:1788572544.931129:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35932:x1875453360444800:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.931168:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x73:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.931295:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35932:x1875453360444800:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 166us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.931303:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4367 00000100:00100000:2.0:1788572544.934699:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.934752:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.934850:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360444928 00000100:00100000:3.0:1788572544.934861:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360444928 00000100:00100000:3.0:1788572544.934927:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360445056 00000100:00100000:3.0:1788572544.934931:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360445056 00000100:00100000:3.0:1788572544.934961:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4368 00000100:00100000:3.0:1788572544.934971:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35931:x1875453360444928:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572544.935035:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6d:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572544.935159:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35931:x1875453360444928:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 188us (471us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.935167:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4368 00000100:00100000:3.0:1788572544.935184:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4369 00000100:00100000:3.0:1788572544.935191:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35930:x1875453360445056:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572544.935321:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6f:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572544.935366:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35930:x1875453360445056:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 175us (614us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.935374:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4369 00000100:00100000:2.0:1788572544.941130:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.941180:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.941329:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360445312 00000100:00100000:0.0:1788572544.941338:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360445312 00000100:00100000:0.0:1788572544.941400:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4370 00000100:00100000:0.0:1788572544.941410:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35932:x1875453360445312:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.941500:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941521:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941536:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941553:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941561:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941571:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941601:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941610:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941616:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.941658:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x73:0x0]: returning size 3 00000004:00200000:0.0:1788572544.941663:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572544.941676:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941689:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941695:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941702:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941707:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.941711:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941719:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941734:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572544.941737:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.941742:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941748:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163478 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941764:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941770:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941774:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941779:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.941782:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941787:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941793:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x11781c1e (ffff9627e7451a00) 00010000:00010000:0.0:1788572544.941795:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.941798:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941804:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16347f lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x11781c1e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941828:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941835:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941842:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941847:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.941850:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941856:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941864:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:0.0:1788572544.941866:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.941869:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.941876:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163486 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572544.941967:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35932:x1875453360445312:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 557us (785us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.941976:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4370 00000100:00100000:3.0:1788572544.942183:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360445184 00000100:00100000:3.0:1788572544.942190:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360445184 00000100:00100000:3.0:1788572544.942242:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1194 00000100:00100000:3.0:1788572544.942250:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2350:x1875453360445184:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572544.942263:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572544.942282:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb16348d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe3e expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572544.942310:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4d:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572544.942327:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=893s v=(0 0) 00010000:00010000:3.0:1788572544.942332:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572544.942343:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb16348d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe3e expref: 49 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.942363:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fb32d297/0x85ace67feb16348d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe3e expref: 49 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.942377:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd600, rc 0) 00000100:00100000:3.0:1788572544.942402:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2350:x1875453360445184:12345-192.168.202.40@tcp:101:diff.0 Request processed in 154us (1285us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.942410:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1194 00000100:00100000:2.0:1788572544.949231:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.949277:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.949318:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.949350:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360445952 00000100:00100000:2.0:1788572544.949360:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360445952 00000100:00100000:2.0:1788572544.949403:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360445824 00000100:00100000:2.0:1788572544.949406:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360445824 00000100:00100000:2.0:1788572544.949420:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360445568 00000100:00100000:2.0:1788572544.949423:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360445568 00000100:00100000:2.0:1788572544.949443:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4371 00000100:00100000:2.0:1788572544.949467:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35931:x1875453360445952:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.949549:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949569:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949583:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949597:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949604:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949611:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949644:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949654:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949661:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.949701:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6d:0x0]: returning size 3 00000004:00200000:2.0:1788572544.949706:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.949721:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949734:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949742:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949749:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949756:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.949761:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949770:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949783:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.949787:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.949792:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949799:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb163494 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949814:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949820:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949827:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949833:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.949835:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949842:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949865:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x1c01e295 (ffff9627e7a95000) 00010000:00010000:2.0:1788572544.949868:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.949871:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949878:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16349b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1c01e295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949894:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949901:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949907:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949913:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.949924:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949931:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949938:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627d55e6400) 00010000:00010000:2.0:1788572544.949940:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.949944:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.949950:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1634a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.950066:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35931:x1875453360445952:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 600us (847us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.950075:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4371 00000100:00100000:2.0:1788572544.950098:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4372 00000100:00100000:2.0:1788572544.950106:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+167:35867:x1875453360445824:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572544.950116:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.950128:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634a9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe4c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950200:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1634b0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950210:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1634b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950219:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1634b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.950228:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xae:0x0]: returning size 3 00010000:00010000:2.0:1788572544.950350:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634a9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe4c expref: 170 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950365:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1634b0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe4c expref: 169 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950372:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:2.0:1788572544.950375:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572544.950377:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572544.950399:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360445824:12345-192.168.202.40@tcp:101:diff.0 Request processed in 295us (1122us total) trans 4294970577 rc 0/0 00000100:00100000:2.0:1788572544.950405:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4372 00000100:00100000:2.0:1788572544.950419:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4373 00000100:00100000:2.0:1788572544.950425:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35932:x1875453360445568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.950433:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.950440:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634be lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe45 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950503:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950512:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950519:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.950526:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x73:0x0]: returning size 3 00010000:00010000:2.0:1788572544.950564:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950571:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950577:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950582:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.950585:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950590:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950599:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:2.0:1788572544.950601:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572544.950605:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950615:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950639:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634be lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe45 expref: 82 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950648:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.950650:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572544.950654:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634be lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe45 expref: 82 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.950662:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572544.950683:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35932:x1875453360445568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 258us (1365us total) trans 4294970578 rc 301/301 00000100:00100000:2.0:1788572544.950689:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4373 00000100:00100000:2.0:1788572544.953988:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.956606:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360446208 00000100:00100000:3.0:1788572544.956634:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360446208 00000100:00100000:3.0:1788572544.956715:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 307 00000100:00100000:3.0:1788572544.956724:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9da050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:2349:x1875453360446208:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572544.956743:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572544.956767:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1634b0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe4c expref: 169 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572544.956789:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffff9627ffc51100) 00010000:00010000:3.0:1788572544.956794:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572544.956865:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9da050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:2349:x1875453360446208:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 142us (2886us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572544.956873:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 307 00000100:00100000:3.0:1788572544.959301:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.959391:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360446080 00000100:00100000:2.0:1788572544.959402:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360446080 00000100:00100000:2.0:1788572544.959470:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4374 00000100:00100000:2.0:1788572544.959480:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35930:x1875453360446080:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.959560:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959579:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959590:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959606:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959614:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959620:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959646:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959656:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959662:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.959703:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6f:0x0]: returning size 3 00000004:00200000:2.0:1788572544.959709:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572544.959723:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959735:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959743:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959751:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959763:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.959767:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959776:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959789:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.959792:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.959796:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959804:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1634d3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959817:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959823:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959830:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959836:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.959838:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959845:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959873:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xa87bcfeb (ffff9627e7a95000) 00010000:00010000:2.0:1788572544.959876:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.959878:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959885:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1634da lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xa87bcfeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959899:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959905:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959911:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959925:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.959928:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959935:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959942:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627d1866400) 00010000:00010000:2.0:1788572544.959945:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572544.959948:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.959955:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572544.960072:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35930:x1875453360446080:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 592us (779us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.960080:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4374 00000100:00100000:2.0:1788572544.964734:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.964780:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.964877:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360446464 00000100:00100000:0.0:1788572544.964887:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360446464 00000100:00100000:0.0:1788572544.964947:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360446592 00000100:00100000:0.0:1788572544.964951:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360446592 00000100:00100000:0.0:1788572544.964975:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4375 00000100:00100000:0.0:1788572544.964984:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35931:x1875453360446464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.965034:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.965058:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1634e8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe53 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965167:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965184:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965211:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.965223:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6d:0x0]: returning size 3 00010000:00010000:0.0:1788572544.965347:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965357:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965364:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965370:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094de00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.965375:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965383:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965398:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627d55e6400) 00010000:00010000:0.0:1788572544.965402:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.965408:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965415:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb1634ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965471:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1634e8 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe53 expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965481:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572544.965483:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572544.965487:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb1634e8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe53 expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.965502:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572544.965561:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35931:x1875453360446464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 578us (842us total) trans 4294970579 rc 301/301 00000100:00100000:0.0:1788572544.965568:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4375 00000100:00100000:0.0:1788572544.965591:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4376 00000100:00100000:0.0:1788572544.965600:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35932:x1875453360446592:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.965619:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x73:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.965689:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35932:x1875453360446592:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 90us (908us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.965697:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4376 00000100:00100000:0.0:1788572544.971741:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.971826:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360447104 00000100:00100000:2.0:1788572544.971836:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360447104 00000100:00100000:2.0:1788572544.971908:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4377 00000100:00100000:2.0:1788572544.971918:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35930:x1875453360447104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.971945:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.971965:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634fd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe61 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972107:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972123:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972136:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572544.972151:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6f:0x0]: returning size 3 00010000:00010000:2.0:1788572544.972263:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972272:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972285:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972292:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572544.972297:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972306:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972318:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627d1866400) 00010000:00010000:2.0:1788572544.972321:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572544.972326:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972333:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163504 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972375:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634fd lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe61 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972384:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572544.972386:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572544.972390:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1634fd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe61 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.972399:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572544.972462:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35930:x1875453360447104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 546us (731us total) trans 4294970580 rc 301/301 00000100:00100000:2.0:1788572544.972474:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4377 00000100:00100000:3.0:1788572544.974124:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.974161:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572544.974193:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.974334:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360446848 00000100:00100000:0.0:1788572544.974344:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360446848 00000100:00100000:0.0:1788572544.974401:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360447232 00000100:00100000:0.0:1788572544.974404:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360447232 00000100:00100000:0.0:1788572544.974427:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4378 00000100:00100000:0.0:1788572544.974435:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35932:x1875453360446848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572544.974466:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572544.974487:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163512 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe5a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974559:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974574:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974585:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974600:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974607:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974614:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974903:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163527 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974917:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163527 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974924:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163527 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572544.974949:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:0.0:1788572544.974953:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572544.974970:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163527 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974983:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974990:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.974997:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975037:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.975042:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975049:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975061:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572544.975064:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.975069:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975079:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163519 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975095:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975102:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975107:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975112:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572544.975115:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975120:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975127:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627e7451a00) 00010000:00010000:0.0:1788572544.975129:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572544.975132:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975137:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163520 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975171:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094de00/0x85ace67feb163512 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe5a expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975189:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163527 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe5a expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572544.975209:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572544.975212:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572544.975215:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:0.0:1788572544.975278:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35932:x1875453360446848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 842us (1116us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.975285:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4378 00000100:00100000:0.0:1788572544.975311:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4379 00000100:00100000:0.0:1788572544.975318:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35931:x1875453360447232:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572544.975337:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6d:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572544.975395:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35931:x1875453360447232:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 77us (1200us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572544.975401:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4379 00000100:00100000:2.0:1788572544.975693:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360446720 00000100:00100000:2.0:1788572544.975700:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360446720 00002000:00100000:2.0:1788572544.975738:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4d:0x0] (0->4095) req@ffff9626c1bced80 x1875453360446720/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:205/0 lens 488/0 e 0 to 0 dl 1788572555 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.975747:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bced80 with x1875453360446720 ext(0->4095) 00002000:00010000:2.0:1788572544.975752:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1bced80 00000100:00100000:2.0:1788572544.975778:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 267 00000100:00100000:2.0:1788572544.975786:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2349:x1875453360446720:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572544.975924:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2349:x1875453360446720:12345-192.168.202.40@tcp:3:diff.0 Request processed in 138us (1810us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572544.975930:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 267 00002000:00100000:2.0:1788572544.975936:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4d:0x0] (0->4095) req@ffff9626c1bced80 x1875453360446720/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:205/0 lens 488/4536 e 0 to 0 dl 1788572555 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572544.975943:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bced80 with x1875453360446720 ext(0->4095) 00002000:00010000:2.0:1788572544.975946:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1bced80 00000100:00100000:0.0:1788572544.981525:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.981596:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360447360 00000100:00100000:2.0:1788572544.981606:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360447360 00000100:00100000:2.0:1788572544.981667:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4380 00000100:00100000:2.0:1788572544.981679:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35930:x1875453360447360:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572544.981714:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6f:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572544.981830:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35930:x1875453360447360:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 153us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572544.981838:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4380 00000100:00100000:3.0:1788572544.987152:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572544.987251:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360447616 00000100:00100000:0.0:1788572544.987262:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360447616 00000100:00100000:0.0:1788572544.987325:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 928 00000100:00100000:0.0:1788572544.987335:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+168:35867:x1875453360447616:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572544.987720:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360447616:12345-192.168.202.40@tcp:35:diff.0 Request processed in 385us (577us total) trans 4294970581 rc 0/0 00000100:00100000:0.0:1788572544.987728:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 928 00000100:00100000:0.0:1788572544.993786:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.993877:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360447872 00000100:00100000:2.0:1788572544.993888:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360447872 00000100:00100000:2.0:1788572544.993962:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4381 00000100:00100000:2.0:1788572544.993972:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35930:x1875453360447872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572544.993999:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572544.994168:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16352e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe68 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572544.994769:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x51:0x0], index 0: 0 00010000:00010000:2.0:1788572544.994957:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163535 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.994973:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163535 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.994986:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163535 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.995077:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16352e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe68 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.995098:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163535 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe68 expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572544.995107:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:2.0:1788572544.995110:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572544.995113:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05a00, rc 0) 00000100:00100000:2.0:1788572544.995186:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35930:x1875453360447872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1216us (1409us total) trans 4294970582 rc 0/0 00000100:00100000:1.0:1788572544.998149:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572544.998509:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4381 00000100:00100000:2.0:1788572544.998555:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360448000 00000100:00100000:2.0:1788572544.998565:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360448000 00000100:00100000:0.0:1788572545.001823:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.002489:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360448256 00000100:00100000:3.0:1788572545.002501:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360448256 00000100:00080000:3.0:1788572545.002532:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788572545.002580:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4382 00000100:00100000:3.0:1788572545.002595:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+169:35867:x1875453360448000:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572545.002621:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.002644:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163543 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe6f expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.002726:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.002742:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.002755:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.002770:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.002776:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.002782:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003323:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb163558 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003334:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb163558 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003343:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb163558 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.003380:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb6:0x0]: returning size 3 00000004:00200000:3.0:1788572545.003385:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.003406:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb163558 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003420:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb163558 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003428:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003434:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003442:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003460:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.003465:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003472:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003485:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:3.0:1788572545.003488:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.003493:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003503:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb16354a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003516:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003522:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003528:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003534:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.003537:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003543:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003551:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xe44515a3 (ffff9627ffc51800) 00010000:00010000:3.0:1788572545.003553:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.003556:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003562:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163551 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003598:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163543 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe6f expref: 171 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003618:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd200/0x85ace67feb163558 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe6f expref: 170 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003625:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627ffc51300) 00010000:00010000:3.0:1788572545.003627:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572545.003630:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd200, rc 0) 00000100:00100000:3.0:1788572545.003699:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360448000:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1111us (5565us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.003708:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4382 00000100:00100000:3.0:1788572545.003735:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4383 00000100:00100000:3.0:1788572545.003744:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35930:x1875453360448256:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.003784:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572545.003798:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb163535 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe68 expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003810:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:3.0:1788572545.003822:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163527 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe5a expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.003836:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.003839:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572545.004218:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004232:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004241:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004254:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004261:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004268:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004398:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004406:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004413:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004419:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.004423:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004430:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004438:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.004441:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.004455:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004463:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16355f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004473:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004479:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004484:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004489:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.004492:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004497:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004505:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xccdd9753 (ffff9627ffc51500) 00010000:00010000:3.0:1788572545.004507:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.004510:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.004515:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163566 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0xccdd9753 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.004565:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35930:x1875453360448256:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 821us (2751us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572545.004573:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4383 00000100:00080000:3.0:1788572545.004601:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788572545.009823:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.009867:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.010088:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.010113:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360448384 00000100:00100000:3.0:1788572545.010123:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360448384 00000100:00100000:3.0:1788572545.010170:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360448768 00000100:00100000:3.0:1788572545.010173:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360448768 00000100:00100000:3.0:1788572545.010187:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4384 00000100:00100000:3.0:1788572545.010209:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35930:x1875453360448384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.010229:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.010252:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16356d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe76 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.010290:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.010305:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.010317:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.010421:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16356d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.010440:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 86 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.010461:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572545.010464:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572545.010469:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572545.010520:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35930:x1875453360448384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 324us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.010528:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4384 00000100:00100000:3.0:1788572545.010547:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4385 00000100:00100000:3.0:1788572545.010555:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35932:x1875453360448768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.010564:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.010572:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16357b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe84 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572545.011115:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x51:0x0], index 1: 0 00010000:00010000:3.0:1788572545.011305:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163582 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.011317:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163582 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.011326:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163582 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.011377:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb16357b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe84 expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.011392:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163582 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe84 expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.011401:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:3.0:1788572545.011404:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572545.011407:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc400, rc 0) 00000100:00100000:3.0:1788572545.011442:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35932:x1875453360448768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 888us (1355us total) trans 4294970583 rc 0/0 00000100:00100000:3.0:1788572545.011461:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4385 00000100:00100000:0.0:1788572545.019481:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360448512 00000100:00100000:0.0:1788572545.019499:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360448512 00000100:00100000:0.0:1788572545.019572:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1195 00000100:00100000:0.0:1788572545.019583:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+36:2350:x1875453360448512:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572545.019601:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.019625:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb163590 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe7d expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572545.019652:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4f:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:0.0:1788572545.019672:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=893s v=(0 0) 00010000:00010000:0.0:1788572545.019677:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.019686:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb163590 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe7d expref: 37 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.019707:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d261d483/0x85ace67feb163590 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe7d expref: 37 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.019721:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094de00, rc 0) 00000100:00100000:0.0:1788572545.019783:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+37:2350:x1875453360448512:12345-192.168.202.40@tcp:101:diff.0 Request processed in 202us (9916us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.019791:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1195 00000100:00100000:0.0:1788572545.020273:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00080000:1.0:1788572545.020344:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572545.020497:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360449408 00000100:00100000:1.0:1788572545.020505:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360449408 00000100:00100000:1.0:1788572545.020576:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4386 00000100:00100000:1.0:1788572545.020587:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35931:x1875453360449408:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.020607:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.020626:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163597 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe92 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572545.021145:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x52:0x0], index 0: 0 00010000:00010000:1.0:1788572545.021513:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16359e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.021532:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16359e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.021544:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16359e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.021610:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163597 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe92 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.021629:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16359e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe92 expref: 89 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.021640:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627c5716600) 00010000:00010000:1.0:1788572545.021643:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572545.021646:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96c00, rc 0) 00000100:00100000:1.0:1788572545.021702:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35931:x1875453360449408:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1118us (1432us total) trans 4294970584 rc 0/0 00000100:00100000:1.0:1788572545.021712:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4386 00000100:00100000:2.0:1788572545.021944:0:6873:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95180 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6873:1875453382371456:0@lo:5:osp-pre-0-0.0 00000100:00100000:2.0:1788572545.021999:0:6873:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.022068:0:6873:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.022100:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.022178:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.022763:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382371456 00000100:00100000:3.0:1788572545.022772:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382371456 00000100:00100000:3.0:1788572545.022820:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1196 00000100:00100000:3.0:1788572545.022829:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:6873:x1875453382371456:12345-0@lo:5:osp-pre-0-0.0 00002000:00080000:3.0:1788572545.022869:0:13272:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0000: reserve 32 objects in group 0x240000400 at 98 00000100:00100000:2.0:1788572545.024309:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360449024 00000100:00100000:2.0:1788572545.024318:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360449024 00000100:00100000:2.0:1788572545.024366:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360449152 00000100:00100000:2.0:1788572545.024369:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360449152 00000100:00100000:2.0:1788572545.024396:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4387 00000100:00100000:2.0:1788572545.024405:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35930:x1875453360449024:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.024497:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.024516:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.024529:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572545.024876:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].69, 88 off13784 00000004:00010000:2.0:1788572545.026462:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000938f68ed reply state 000000003426a59f transno 4294970585 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.026478:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572545.026593:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35930:x1875453360449024:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2183us (4487us total) trans 4294970585 rc 0/0 00000100:00100000:2.0:1788572545.026603:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4387 00000100:00100000:2.0:1788572545.026636:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4388 00000100:00100000:2.0:1788572545.026647:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35932:x1875453360449152:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.026663:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.026678:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1635b3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe8b expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.026737:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.026752:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.026763:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.026778:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.026785:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.026791:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027414:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1635c8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027426:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1635c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027434:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1635c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.027474:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:2.0:1788572545.027478:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.027495:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1635c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027508:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027514:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027520:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027527:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.027532:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027539:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027553:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572545.027557:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.027561:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027567:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1635ba lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027584:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027591:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027597:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027603:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.027605:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027611:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027619:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627e7a95300) 00010000:00010000:2.0:1788572545.027621:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.027624:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027629:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1635c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027663:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1635b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe8b expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027676:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1635c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe8b expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.027692:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572545.027694:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572545.027697:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572545.027733:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35932:x1875453360449152:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1090us (5555us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.027740:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4388 00000100:00100000:1.0:1788572545.031455:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572545.035674:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.035692:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.035701:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.035707:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.035712:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.035721:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.035737:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:2.0:1788572545.035740:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572545.035745:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 2/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.035751:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb1635ac lrc: 0/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.035896:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360449536 00000100:00100000:0.0:1788572545.035908:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360449536 00000100:00100000:0.0:1788572545.035975:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4389 00000100:00100000:0.0:1788572545.035985:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35930:x1875453360449536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.036028:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.036049:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 2/0,0 mode: --/CR res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efe99 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.036108:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.036122:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.036140:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572545.036143:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572545.036146:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:0.0:1788572545.036220:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35930:x1875453360449536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 236us (4786us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.036227:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4389 00000100:00100000:1.0:1788572545.050604:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.050687:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.055987:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.058958:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450048 00000100:00100000:2.0:1788572545.058971:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450048 00000100:00100000:2.0:1788572545.059053:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360449792 00000100:00100000:2.0:1788572545.059057:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360449792 00000100:00100000:2.0:1788572545.059087:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4390 00000100:00100000:2.0:1788572545.059097:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360450048:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572545.059118:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.059139:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1635d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efea0 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059364:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1635dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059381:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1635dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059395:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1635dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.059407:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb6:0x0]: returning size 3 00010000:00010000:2.0:1788572545.059541:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1635d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efea0 expref: 173 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059562:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1635dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efea0 expref: 172 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059571:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627ffc51300) 00010000:00010000:2.0:1788572545.059574:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572545.059578:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572545.059648:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:35867:x1875453360450048:12345-192.168.202.40@tcp:101:diff.0 Request processed in 552us (9056us total) trans 4294970586 rc 0/0 00000100:00100000:2.0:1788572545.059658:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4390 00000100:00100000:2.0:1788572545.059680:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4391 00000100:00100000:2.0:1788572545.059688:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35932:x1875453360449792:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.059718:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572545.059734:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163582 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe84 expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059747:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627ffc51000) 00010000:00010000:2.0:1788572545.059762:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb1635c8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe8b expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059771:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572545.059775:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572545.059789:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059798:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059806:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059821:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059828:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.059836:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060210:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060219:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060227:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060233:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.060238:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060246:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060256:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572545.060259:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.060263:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060270:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1635eb lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060280:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060286:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060293:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060298:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.060301:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060308:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060314:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x7d00de53 (ffff9627e7a95c00) 00010000:00010000:2.0:1788572545.060316:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.060319:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.060326:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb1635f2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x7d00de53 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.060371:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35932:x1875453360449792:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 683us (9683us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572545.060378:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4391 00000100:00100000:3.0:1788572545.061566:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:6873:x1875453382371456:12345-0@lo:5:osp-pre-0-0.0 Request processed in 38734us (39562us total) trans 4294967619 rc 0/0 00000100:00100000:3.0:1788572545.061576:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1196 00000100:00100000:2.0:1788572545.061718:0:6873:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95180 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6873:1875453382371456:0@lo:5:osp-pre-0-0.0 00000004:00080000:2.0:1788572545.061733:0:6873:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0000-osc-MDT0000: current precreated pool: [0x240000400:0x52:0x0]-[0x240000400:0x81:0x0] 00000100:00100000:1.0:1788572545.062058:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.066869:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.066905:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.066934:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.067563:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450304 00000100:00100000:2.0:1788572545.067574:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450304 00000100:00100000:2.0:1788572545.067629:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1197 00000100:00100000:2.0:1788572545.067639:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360450304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572545.068100:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360450304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 461us (6042us total) trans 4294967620 rc 0/0 00000100:00100000:2.0:1788572545.068108:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1197 00000100:00100000:2.0:1788572545.068133:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450688 00000100:00100000:2.0:1788572545.068136:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450688 00002000:00100000:2.0:1788572545.068159:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4f:0x0] (0->4095) req@ffff9627d0fb2a00 x1875453360450688/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.068169:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d0fb2a00 with x1875453360450688 ext(0->4095) 00002000:00010000:2.0:1788572545.068174:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d0fb2a00 00000100:00100000:2.0:1788572545.068189:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 268 00000100:00100000:2.0:1788572545.068597:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+37:2350:x1875453360450688:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572545.068804:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450176 00000100:00100000:3.0:1788572545.068809:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450176 00000100:00100000:3.0:1788572545.068839:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450432 00000100:00100000:3.0:1788572545.068841:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450432 00000100:00100000:3.0:1788572545.068857:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4392 00000100:00100000:3.0:1788572545.068864:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35931:x1875453360450176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.068877:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.068895:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1635f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efea7 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.068963:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.068977:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.068994:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069032:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069039:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069046:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069314:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16360e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069323:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16360e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069330:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16360e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.069353:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:3.0:1788572545.069356:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.069370:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16360e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069380:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069386:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069397:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069403:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.069407:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069415:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069426:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572545.069429:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.069433:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069438:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163600 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069460:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069466:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069471:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069476:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.069479:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069489:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069496:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627ffc51500) 00010000:00010000:3.0:1788572545.069498:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.069501:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069507:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163607 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069537:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cdc00/0x85ace67feb1635f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efea7 expref: 93 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069550:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16360e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efea7 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.069558:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.069560:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572545.069564:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc800, rc 0) 00000100:00100000:2.0:1788572545.069637:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+37:2350:x1875453360450688:12345-192.168.202.40@tcp:3:diff.0 Request processed in 1440us (2700us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572545.069646:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 268 00002000:00100000:2.0:1788572545.069654:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4f:0x0] (0->4095) req@ffff9627d0fb2a00 x1875453360450688/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.069663:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d0fb2a00 with x1875453360450688 ext(0->4095) 00002000:00010000:2.0:1788572545.069667:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d0fb2a00 00000100:00100000:2.0:1788572545.069696:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450560 00000100:00100000:2.0:1788572545.069700:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450560 00000100:00100000:2.0:1788572545.069732:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 308 00000100:00100000:2.0:1788572545.069738:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:2348:x1875453360450560:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.069746:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.069759:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1635dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efea0 expref: 172 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.069771:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffff9627ffc51300) 00010000:00010000:2.0:1788572545.069776:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.070721:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4393 00000100:00100000:0.0:1788572545.070738:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35932:x1875453360450432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.070752:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.070770:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163615 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efeae expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.070795:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.070806:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.070820:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.070910:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163615 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.070930:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 93 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.070938:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572545.070940:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572545.070943:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d600, rc 0) 00000100:00100000:3.0:1788572545.072845:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35931:x1875453360450176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3965us (16850us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.072861:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4392 00000100:00100000:2.0:1788572545.073070:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:2348:x1875453360450560:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3328us (6162us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.073082:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 308 00000100:00100000:0.0:1788572545.073231:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35932:x1875453360450432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2498us (6380us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.073239:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4393 00000100:00100000:1.0:1788572545.078111:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.078538:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360450944 00000100:00100000:0.0:1788572545.078549:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360450944 00000100:00100000:0.0:1788572545.078613:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 929 00000100:00100000:0.0:1788572545.078624:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35930:x1875453360450944:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.079391:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35930:x1875453360450944:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 765us (1291us total) trans 4294970587 rc 0/0 00000100:00100000:0.0:1788572545.079401:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 929 00000100:00100000:0.0:1788572545.083689:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.083777:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.083809:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.083855:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360451456 00000100:00100000:3.0:1788572545.083866:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360451456 00000100:00100000:3.0:1788572545.083923:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 930 00000100:00100000:3.0:1788572545.083934:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360451456:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572545.084908:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:35867:x1875453360451456:12345-192.168.202.40@tcp:35:diff.0 Request processed in 971us (1125us total) trans 4294970588 rc 0/0 00000100:00100000:3.0:1788572545.084928:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 930 00000100:00100000:1.0:1788572545.089454:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360451200 00000100:00100000:1.0:1788572545.089472:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360451200 00000100:00100000:3.0:1788572545.090033:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360451712 00000100:00100000:3.0:1788572545.090039:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360451712 00000100:00100000:3.0:1788572545.090100:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4394 00000100:00100000:3.0:1788572545.090110:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35931:x1875453360451712:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.090169:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572545.090374:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16359e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe92 expref: 94 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090397:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627c5716600) 00010000:00010000:3.0:1788572545.090417:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16360e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efea7 expref: 93 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090427:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.090431:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572545.090464:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090478:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090490:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090506:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090519:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090526:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090735:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090744:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090751:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090757:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.090762:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090770:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090780:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.090783:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.090788:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090794:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb163623 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090803:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090810:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090816:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090821:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.090824:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090830:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.090837:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xb124002e (ffff9627ffc51800) 00010000:00010000:3.0:1788572545.090839:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.090842:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.091164:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4395 00000100:00100000:0.0:1788572545.091174:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35932:x1875453360451200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.091247:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.091258:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.091267:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572545.091611:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].67, 88 off13680 00000004:00080000:0.0:1788572545.091622:0:5839:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 893160616971ns, next at 894160613611ns, rc = 0 00000004:00010000:0.0:1788572545.091706:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002f72e0aa reply state 000000009928e36a transno 4294970589 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.091716:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572545.092297:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16362a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0xb124002e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.092658:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35932:x1875453360451200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1481us (8977us total) trans 4294970589 rc 0/0 00000100:00100000:0.0:1788572545.092667:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4395 00000100:00100000:3.0:1788572545.092743:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35931:x1875453360451712:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2634us (8933us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572545.092750:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4394 00000100:00100000:2.0:1788572545.092823:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.092934:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360451968 00000100:00100000:0.0:1788572545.092941:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360451968 00000100:00100000:0.0:1788572545.092978:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4396 00000100:00100000:0.0:1788572545.092986:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35930:x1875453360451968:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.093068:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35930:x1875453360451968:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 82us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.093075:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4396 00000100:00100000:2.0:1788572545.098260:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.098407:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360452096 00000100:00100000:0.0:1788572545.098418:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360452096 00000100:00100000:0.0:1788572545.098487:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4397 00000100:00100000:0.0:1788572545.098498:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:35867:x1875453360452096:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572545.098523:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.098546:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163638 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efeb5 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.098623:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.098639:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.098652:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.098667:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.098675:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.098681:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099046:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16364d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099056:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16364d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099064:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16364d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.099104:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb5:0x0]: returning size 3 00000004:00200000:0.0:1788572545.099109:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572545.099128:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16364d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099141:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16364d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099150:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099157:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099164:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099170:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.099175:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099183:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099195:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572545.099198:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.099203:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099210:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16363f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099232:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099239:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099245:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099251:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.099254:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099260:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099267:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x2c9ecea3 (ffff9627e7451e00) 00010000:00010000:0.0:1788572545.099269:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.099272:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099279:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163646 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099315:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163638 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeb5 expref: 174 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099328:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ce00/0x85ace67feb16364d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeb5 expref: 173 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.099342:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627e7451500) 00010000:00010000:0.0:1788572545.099345:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572545.099348:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ce00, rc 0) 00000100:00100000:0.0:1788572545.099401:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360452096:12345-192.168.202.40@tcp:101:diff.0 Request processed in 905us (1151us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.099409:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4397 00000100:00100000:0.0:1788572545.107468:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.107507:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.107610:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572545.107696:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.107711:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.107720:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.107727:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094ca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.107733:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.107742:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.107763:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:1.0:1788572545.107766:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572545.107772:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.107784:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb163631 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.107873:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.107895:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360452608 00000100:00100000:0.0:1788572545.107905:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360452608 00000100:00100000:0.0:1788572545.107964:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1198 00000100:00100000:0.0:1788572545.107974:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+37:2347:x1875453360452608:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572545.107991:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.108034:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb163654 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efed1 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572545.108063:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4c:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:0.0:1788572545.108086:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=893s v=(0 0) 00010000:00010000:0.0:1788572545.108091:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.108102:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb163654 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efed1 expref: 38 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.108119:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000472f5f3d/0x85ace67feb163654 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efed1 expref: 38 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.108132:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47800, rc 0) 00000100:00100000:0.0:1788572545.108182:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+38:2347:x1875453360452608:12345-192.168.202.40@tcp:101:diff.0 Request processed in 209us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.108190:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1198 00000100:00100000:3.0:1788572545.108455:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360452224 00000100:00100000:3.0:1788572545.108460:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360452224 00000100:00100000:3.0:1788572545.108498:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360452352 00000100:00100000:3.0:1788572545.108502:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360452352 00000100:00100000:3.0:1788572545.108517:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360452480 00000100:00100000:3.0:1788572545.108519:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360452480 00000100:00100000:3.0:1788572545.108537:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4398 00000100:00100000:3.0:1788572545.108543:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35931:x1875453360452224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.108554:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.108569:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb16365b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efebc expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108601:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108613:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108623:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108720:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cce00/0x85ace67feb16365b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108735:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108742:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:3.0:1788572545.108745:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572545.108748:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc800, rc 0) 00000100:00100000:3.0:1788572545.108779:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35931:x1875453360452224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 235us (1332us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.108785:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4398 00000100:00100000:3.0:1788572545.108849:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4399 00000100:00100000:3.0:1788572545.108856:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35932:x1875453360452352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.108869:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.108879:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efec3 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108907:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108916:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.108931:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:3.0:1788572545.108933:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572545.108936:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572545.108958:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35932:x1875453360452352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 103us (1451us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.108965:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4399 00000100:00100000:3.0:1788572545.108979:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4400 00000100:00100000:3.0:1788572545.108986:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35930:x1875453360452480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.108994:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.109026:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163670 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efeca expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.109069:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163677 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.109079:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163677 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.109089:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163677 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.109117:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.109130:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163677 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.109151:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163670 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeca expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.109164:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163677 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeca expref: 95 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.109172:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572545.109174:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572545.109177:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572545.113584:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35930:x1875453360452480:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4580us (5955us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.113600:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4400 00000100:00100000:3.0:1788572545.113696:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.113755:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.113817:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360452864 00000100:00100000:3.0:1788572545.113825:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360452864 00000100:00100000:3.0:1788572545.113855:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360453120 00000100:00100000:3.0:1788572545.113857:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360453120 00000100:00100000:3.0:1788572545.113872:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4401 00000100:00100000:3.0:1788572545.113880:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360452864:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572545.113901:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.113928:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16367e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efed8 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114046:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163685 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114062:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163685 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114074:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163685 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.114086:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb5:0x0]: returning size 3 00010000:00010000:3.0:1788572545.114231:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16367e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efed8 expref: 176 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114252:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163685 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efed8 expref: 175 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114262:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627e7451500) 00010000:00010000:3.0:1788572545.114266:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572545.114270:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc 0) 00000100:00100000:3.0:1788572545.114319:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360452864:12345-192.168.202.40@tcp:101:diff.0 Request processed in 439us (623us total) trans 4294970590 rc 0/0 00000100:00100000:3.0:1788572545.114326:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4401 00000100:00100000:3.0:1788572545.114363:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4402 00000100:00100000:3.0:1788572545.114371:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35931:x1875453360453120:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.114425:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114434:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.114441:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572545.114783:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].70, 88 off13872 00000004:00080000:3.0:1788572545.114792:0:9040:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 893183787026ns, next at 894183784927ns, rc = 0 00000004:00010000:3.0:1788572545.114879:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a0d0439f reply state 000000009d8c9385 transno 4294970591 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.114889:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572545.114945:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35931:x1875453360453120:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 574us (1189us total) trans 4294970591 rc 0/0 00000100:00100000:3.0:1788572545.114952:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4402 00000100:00100000:3.0:1788572545.120838:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.120928:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.121213:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360453376 00000100:00100000:0.0:1788572545.121224:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360453376 00002000:00100000:0.0:1788572545.121271:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4c:0x0] (0->4095) req@ffff9627fb30f800 x1875453360453376/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572545.121284:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30f800 with x1875453360453376 ext(0->4095) 00002000:00010000:0.0:1788572545.121290:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30f800 00000100:00100000:0.0:1788572545.121317:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 269 00000100:00100000:0.0:1788572545.121326:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+38:2350:x1875453360453376:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572545.121543:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+38:2350:x1875453360453376:12345-192.168.202.40@tcp:3:diff.0 Request processed in 217us (713us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572545.121552:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 269 00002000:00100000:0.0:1788572545.121559:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4c:0x0] (0->4095) req@ffff9627fb30f800 x1875453360453376/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572545.121567:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30f800 with x1875453360453376 ext(0->4095) 00002000:00010000:0.0:1788572545.121571:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30f800 00010000:00010000:2.0:1788572545.121908:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.121930:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.121939:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.121947:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1679400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.121952:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.121962:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.121979:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:2.0:1788572545.121982:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572545.121988:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.121995:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679400/0x85ace67feb163693 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.122416:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360453248 00000100:00100000:1.0:1788572545.122423:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360453248 00000100:00100000:1.0:1788572545.122501:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1199 00000100:00100000:1.0:1788572545.122512:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360453248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.122525:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.122547:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efedf expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572545.122570:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x51:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7380, blocks: 0x1 00010000:00010000:1.0:1788572545.122586:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=893s v=(0 0) 00010000:00010000:1.0:1788572545.122591:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.122600:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.122617:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.122629:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572545.122676:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360453248:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 166us (1746us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.122683:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1199 00000100:00100000:3.0:1788572545.125837:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.125890:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.126041:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360453632 00000100:00100000:0.0:1788572545.126053:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360453632 00000100:00100000:0.0:1788572545.126113:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4403 00000100:00100000:0.0:1788572545.126124:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35931:x1875453360453632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.126149:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.126173:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efee6 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.126257:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 97 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.126273:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 97 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.126294:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:0.0:1788572545.126297:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572545.126301:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:0.0:1788572545.126354:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35931:x1875453360453632:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 231us (525us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.126362:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4403 00000100:00100000:1.0:1788572545.126848:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360453504 00000100:00100000:1.0:1788572545.126855:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360453504 00000100:00100000:1.0:1788572545.126926:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 309 00000100:00100000:1.0:1788572545.126936:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:2347:x1875453360453504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572545.126947:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572545.126967:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163685 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efed8 expref: 175 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.126984:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffff9627e7451500) 00010000:00010000:1.0:1788572545.126988:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572545.127142:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:2347:x1875453360453504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 207us (1250us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.127150:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 309 00000100:00100000:3.0:1788572545.131700:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.132996:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360453888 00000100:00100000:0.0:1788572545.133051:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360453888 00000100:00100000:0.0:1788572545.133121:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 931 00000100:00100000:0.0:1788572545.133133:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:35867:x1875453360453888:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572545.133553:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360453888:12345-192.168.202.40@tcp:35:diff.0 Request processed in 421us (1862us total) trans 4294970592 rc 0/0 00000100:00100000:0.0:1788572545.133562:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 931 00000100:00100000:2.0:1788572545.135864:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.136041:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360454016 00000100:00100000:1.0:1788572545.136050:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360454016 00000100:00100000:1.0:1788572545.136117:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1200 00000100:00100000:1.0:1788572545.136129:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360454016:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.136433:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360454016:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 306us (588us total) trans 4294967621 rc 0/0 00000100:00100000:1.0:1788572545.136442:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1200 00000100:00100000:1.0:1788572545.140107:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.140156:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.140268:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360454272 00000100:00100000:0.0:1788572545.140280:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360454272 00000100:00100000:0.0:1788572545.140343:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360454400 00000100:00100000:0.0:1788572545.140347:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360454400 00000100:00100000:0.0:1788572545.140379:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4404 00000100:00100000:0.0:1788572545.140390:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35930:x1875453360454272:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.140518:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35930:x1875453360454272:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 129us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.140526:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4404 00000100:00100000:0.0:1788572545.140546:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4405 00000100:00100000:0.0:1788572545.140553:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360454400:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572545.140564:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.140588:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1636a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efeed expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.140649:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.140667:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.140680:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.140695:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.140703:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.140710:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141051:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1636bd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141062:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1636bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141069:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1636bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.141107:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb4:0x0]: returning size 3 00000004:00200000:0.0:1788572545.141112:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572545.141132:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1636bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141143:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1636bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141150:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141157:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141163:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141170:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.141174:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141183:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141196:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572545.141200:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.141205:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141212:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb1636af lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141228:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141235:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141241:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141245:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.141248:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141254:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141261:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x74f887a3 (ffff9627e7451e00) 00010000:00010000:0.0:1788572545.141263:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.141266:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141272:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1636b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141308:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1636a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeed expref: 177 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141323:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094ca00/0x85ace67feb1636bd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeed expref: 176 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.141331:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:0.0:1788572545.141333:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572545.141336:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094ca00, rc 0) 00000100:00100000:0.0:1788572545.141365:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:35867:x1875453360454400:12345-192.168.202.40@tcp:101:diff.0 Request processed in 813us (1210us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.141372:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4405 00010000:00010000:2.0:1788572545.146310:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572545.147937:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.149579:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360454528 00000100:00100000:0.0:1788572545.149596:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360454528 00000100:00100000:0.0:1788572545.149692:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1201 00000100:00100000:0.0:1788572545.149703:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35930:x1875453360454528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.149723:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.149745:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efef4 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.149784:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.149793:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.149798:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.149805:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.149865:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.149881:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.149901:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382371584:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572545.150049:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b799c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.153496:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.153537:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.153679:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360454912 00000100:00100000:3.0:1788572545.153689:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360454912 00000100:00100000:3.0:1788572545.153750:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 932 00000100:00100000:3.0:1788572545.153760:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35931:x1875453360454912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.154387:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35931:x1875453360454912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 626us (847us total) trans 4294970593 rc 0/0 00000100:00100000:3.0:1788572545.154397:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 932 00000100:00100000:1.0:1788572545.154647:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360454656 00000100:00100000:1.0:1788572545.154653:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360454656 00000100:00100000:1.0:1788572545.154711:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1202 00000100:00100000:1.0:1788572545.154722:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+38:2350:x1875453360454656:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572545.154734:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.154760:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb1636cb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efefb expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572545.154789:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4b:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:1.0:1788572545.154808:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=893s v=(0 0) 00010000:00010000:1.0:1788572545.154812:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.154822:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb1636cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efefb expref: 39 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.154841:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e3efcdf6/0x85ace67feb1636cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efefb expref: 39 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.154866:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97e00, rc 0) 00000100:00100000:1.0:1788572545.154895:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+39:2350:x1875453360454656:12345-192.168.202.40@tcp:101:diff.0 Request processed in 176us (1410us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.154902:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1202 00000100:00100000:1.0:1788572545.157319:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.157489:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360455040 00000100:00100000:0.0:1788572545.157499:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360455040 00000100:00100000:0.0:1788572545.157564:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1203 00000100:00100000:0.0:1788572545.157574:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360455040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.157883:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360455040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 308us (572us total) trans 4294967620 rc 0/0 00000100:00100000:0.0:1788572545.157891:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1203 00000100:00100000:2.0:1788572545.158160:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382371584:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572545.158192:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.158214:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad600, rc 0) 00000100:00100000:2.0:1788572545.158238:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35930:x1875453360454528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8537us (10321us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.158245:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1201 00000100:00100000:0.0:1788572545.160369:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.160484:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360455168 00000100:00100000:2.0:1788572545.160495:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360455168 00010000:00010000:2.0:1788572545.160539:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.160584:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1043 00000100:00100000:2.0:1788572545.160594:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360455168:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.160610:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.160635:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.160649:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.160678:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb16369a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efedf expref: 14 pid: 6682 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.160694:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572545.160698:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572545.160707:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.160717:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.160757:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.160771:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382371712:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572545.160835:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f240 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.166896:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.167610:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.167643:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.167674:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360455808 00000100:00100000:0.0:1788572545.167684:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360455808 00000100:00100000:0.0:1788572545.167741:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4406 00000100:00100000:0.0:1788572545.167752:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35931:x1875453360455808:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.167859:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35931:x1875453360455808:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 108us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.167867:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4406 00000100:00100000:2.0:1788572545.168108:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382371712:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572545.168132:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.168158:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360455168:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7566us (7799us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.168167:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1043 00000100:00100000:1.0:1788572545.168282:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360455552 00000100:00100000:1.0:1788572545.168287:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360455552 00002000:00100000:1.0:1788572545.168333:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x51:0x0] (0->2) req@ffff9627c9a58700 x1875453360455552/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.168344:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a58700 with x1875453360455552 ext(0->2) 00002000:00010000:1.0:1788572545.168349:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c9a58700 00000100:00100000:1.0:1788572545.168376:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 270 00000100:00100000:1.0:1788572545.168386:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360455552:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.168825:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360455552:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 439us (1215us total) trans 4294967622 rc 0/0 00000100:00100000:1.0:1788572545.168833:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 270 00002000:00100000:1.0:1788572545.168840:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x51:0x0] (0->2) req@ffff9627c9a58700 x1875453360455552/t4294967622(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/448 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.168849:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9a58700 with x1875453360455552 ext(0->2) 00002000:00010000:1.0:1788572545.168853:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c9a58700 00000100:00100000:3.0:1788572545.169615:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360455424 00000100:00100000:3.0:1788572545.169624:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360455424 00000100:00100000:3.0:1788572545.169681:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 933 00000100:00100000:3.0:1788572545.169691:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35932:x1875453360455424:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.170422:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35932:x1875453360455424:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 730us (3534us total) trans 4294970594 rc 0/0 00000100:00100000:3.0:1788572545.170432:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 933 00000100:00100000:0.0:1788572545.176542:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.176585:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.176658:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360455936 00000100:00100000:2.0:1788572545.176668:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360455936 00000100:00100000:2.0:1788572545.176730:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360456064 00000100:00100000:2.0:1788572545.176734:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360456064 00000100:00100000:2.0:1788572545.176762:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4407 00000100:00100000:2.0:1788572545.176773:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35930:x1875453360455936:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.177075:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4408 00000100:00100000:3.0:1788572545.177085:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35931:x1875453360456064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.177096:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.177118:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1636d2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff02 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.177183:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1636d9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.177208:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1636d9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.177220:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1636d9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.177253:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.177269:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1636d9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.177295:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678200/0x85ace67feb1636d2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff02 expref: 102 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.177313:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1636d9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff02 expref: 101 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.177321:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:3.0:1788572545.177324:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572545.177328:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679800, rc 0) 00000100:00100000:3.0:1788572545.177379:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35931:x1875453360456064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 296us (792us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.177385:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4408 00000100:00100000:0.0:1788572545.194842:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.195174:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.201623:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.201657:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360456704 00000100:00100000:2.0:1788572545.201667:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360456704 00000100:00100000:2.0:1788572545.201721:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1204 00000100:00100000:2.0:1788572545.201731:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360456704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.201752:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.201774:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff10 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572545.201804:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x52:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7380, blocks: 0x1 00010000:00010000:2.0:1788572545.201826:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=893s v=(0 0) 00010000:00010000:2.0:1788572545.201831:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572545.201840:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 16 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.201860:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 16 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.201874:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572545.201937:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360456704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 207us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.201945:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1204 00000100:00100000:1.0:1788572545.203522:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360456576 00000100:00100000:1.0:1788572545.203535:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360456576 00000100:00100000:1.0:1788572545.203641:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360456320 00000100:00100000:1.0:1788572545.203645:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360456320 00000100:00100000:1.0:1788572545.203678:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4409 00000100:00100000:1.0:1788572545.203689:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:35867:x1875453360456576:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572545.203712:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.203731:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1636e7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff09 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.203839:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1636ee lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.203854:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1636ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.203866:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1636ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572545.203879:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb4:0x0]: returning size 3 00010000:00010000:1.0:1788572545.204093:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1636e7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff09 expref: 179 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.204118:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1636ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff09 expref: 178 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.204127:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:1.0:1788572545.204130:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572545.204134:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572545.204193:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360456576:12345-192.168.202.40@tcp:101:diff.0 Request processed in 506us (9366us total) trans 4294970595 rc 0/0 00000100:00100000:1.0:1788572545.204216:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4409 00000100:00100000:1.0:1788572545.204235:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4410 00000100:00100000:1.0:1788572545.204246:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35932:x1875453360456320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.204293:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35932:x1875453360456320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 50us (9120us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.204300:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4410 00000100:00100000:2.0:1788572545.207266:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.211226:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360456960 00000100:00100000:2.0:1788572545.211243:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360456960 00000100:00100000:2.0:1788572545.211321:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4411 00000100:00100000:2.0:1788572545.211332:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35931:x1875453360456960:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572545.211457:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35931:x1875453360456960:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 125us (4201us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.211465:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4411 00000100:00100000:2.0:1788572545.222055:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.222089:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457088 00000100:00100000:2.0:1788572545.222099:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457088 00000100:00100000:2.0:1788572545.222156:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 310 00000100:00100000:2.0:1788572545.222165:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:2350:x1875453360457088:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.222181:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.222217:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1636ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff09 expref: 178 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222235:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffff9627fb1e7900) 00010000:00010000:2.0:1788572545.222241:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.222287:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:2350:x1875453360457088:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 123us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.222295:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 310 00000100:00100000:2.0:1788572545.222338:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.222360:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457216 00000100:00100000:2.0:1788572545.222364:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457216 00000100:00100000:2.0:1788572545.222391:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4412 00000100:00100000:2.0:1788572545.222397:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35932:x1875453360457216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.222414:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.222431:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1636fc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff17 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222516:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163703 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222532:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163703 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222544:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163703 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.222585:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.222605:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163703 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222630:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1636fc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff17 expref: 103 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222646:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163703 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff17 expref: 102 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.222655:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:2.0:1788572545.222658:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572545.222661:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00000100:00100000:2.0:1788572545.222684:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35932:x1875453360457216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 287us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.222692:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4412 00000100:00100000:2.0:1788572545.222781:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.222802:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457344 00000100:00100000:2.0:1788572545.222805:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457344 00002000:00100000:2.0:1788572545.222832:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4b:0x0] (0->4095) req@ffff9626c1b5ad80 x1875453360457344/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.222843:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b5ad80 with x1875453360457344 ext(0->4095) 00002000:00010000:2.0:1788572545.222848:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b5ad80 00000100:00100000:2.0:1788572545.222866:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 271 00000100:00100000:2.0:1788572545.222873:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+39:2349:x1875453360457344:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572545.223056:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+39:2349:x1875453360457344:12345-192.168.202.40@tcp:3:diff.0 Request processed in 182us (274us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572545.223064:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 271 00002000:00100000:2.0:1788572545.223070:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4b:0x0] (0->4095) req@ffff9626c1b5ad80 x1875453360457344/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.223079:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b5ad80 with x1875453360457344 ext(0->4095) 00002000:00010000:2.0:1788572545.223083:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b5ad80 00000100:00100000:2.0:1788572545.228344:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.228377:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.228454:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457600 00000100:00100000:0.0:1788572545.228467:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457600 00000100:00100000:0.0:1788572545.228522:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457472 00000100:00100000:0.0:1788572545.228525:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457472 00000100:00100000:0.0:1788572545.228554:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1205 00000100:00100000:0.0:1788572545.228565:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360457600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.228585:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.228611:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff25 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572545.228642:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x51:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7380, blocks: 0x1 00010000:00010000:0.0:1788572545.228661:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=893s v=(0 0) 00010000:00010000:0.0:1788572545.228665:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.228675:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.228695:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.228715:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:0.0:1788572545.228767:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360457600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 203us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.228775:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1205 00000100:00100000:0.0:1788572545.228793:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1206 00000100:00100000:0.0:1788572545.228800:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35931:x1875453360457472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.228807:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.228818:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff1e expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.228838:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 17 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.228845:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.228848:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572545.228852:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 17 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.228891:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 17 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.228905:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 17 pid: 10416 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.228940:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382371840:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572545.228991:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628008d8900 going to sleep for 11 seconds 00000001:00080000:2.0:1788572545.230750:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970592 is committed 00000001:00080000:2.0:1788572545.230770:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970589 is committed 00000004:00080000:2.0:1788572545.230775:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 38 committed 00000020:00080000:2.0:1788572545.230813:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ec075f00 refcount 2 committed 0 result 0 batchid 125 00000020:00080000:2.0:1788572545.230819:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7e600 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ec075480 00000020:00080000:2.0:1788572545.230822:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7e300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c375d800 00000020:00080000:2.0:1788572545.230841:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d12b0a80 refcount 2 committed 0 result 0 batchid 124 00000020:00080000:2.0:1788572545.230845:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7e900 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d12b0c00 00000020:00080000:2.0:1788572545.230848:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7ec00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4a7cc00 00000020:00080000:2.0:1788572545.230893:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c7e79540 refcount 2 committed 0 result 0 batchid 126 00000020:00080000:2.0:1788572545.230897:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3600 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ca85e840 00000020:00080000:2.0:1788572545.230901:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fe3a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff962800917400 00000001:00080000:2.0:1788572545.230931:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970594 is committed 00000004:00080000:2.0:1788572545.230938:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 43 committed 00000020:00080000:2.0:1788572545.230969:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 123 batchid updated 00000020:00080000:2.0:1788572545.230981:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 126 committed batchid 123 00000020:00080000:2.0:1788572545.231184:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 123 00000100:00100000:2.0:1788572545.231423:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382371968:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572545.231510:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.231555:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572545.231561:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382372096:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572545.231583:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.231612:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382372224:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572545.231635:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.231654:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1788572545.231659:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382372352:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572545.231678:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.231708:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382371968 00000100:00100000:2.0:1788572545.231716:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382371968 00000100:00100000:2.0:1788572545.231751:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372096 00000100:00100000:2.0:1788572545.231754:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372096 00000100:00100000:2.0:1788572545.231764:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372224 00000100:00100000:2.0:1788572545.231766:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372224 00000100:00100000:2.0:1788572545.231774:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372352 00000100:00100000:2.0:1788572545.231777:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372352 00000100:00100000:2.0:1788572545.231789:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1207 00000100:00100000:2.0:1788572545.231797:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382371968:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572545.231960:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572545.231964:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb mtime from disk: 1788572516 -> 1788572516 00002000:00010000:2.0:1788572545.231966:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb atime from disk: 1788572516 -> 1788572516 00002000:00010000:2.0:1788572545.231969:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb ctime from disk: 1788572527 -> 1788572527 00002000:00010000:2.0:1788572545.231972:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4f:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572545.232038:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3302:x1875453382371968:12345-0@lo:2:osp-syn-0-0.0 Request processed in 241us (528us total) trans 4294967623 rc 0/0 00000100:00100000:2.0:1788572545.232047:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1207 00000100:00100000:2.0:1788572545.232067:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1208 00000100:00100000:2.0:1788572545.232074:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382372096:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572545.232163:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3302:x1875453382372096:12345-0@lo:2:osp-syn-0-0.0 Request processed in 90us (579us total) trans 4294967624 rc 0/0 00000100:00100000:2.0:1788572545.232170:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1208 00000100:00100000:2.0:1788572545.232183:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1209 00000100:00100000:2.0:1788572545.232190:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382372224:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572545.232334:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1210 00000100:00100000:1.0:1788572545.232358:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382372352:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:1.0:1788572545.232432:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788572545.232436:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb mtime from disk: 0 -> 0 00002000:00010000:1.0:1788572545.232438:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572545.232440:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb ctime from disk: 1788572544 -> 1788572544 00002000:00010000:1.0:1788572545.232442:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:1.0:1788572545.232490:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382372352:12345-0@lo:2:osp-syn-0-0.0 Request processed in 148us (811us total) trans 4294967625 rc 0/0 00000100:00100000:1.0:1788572545.232497:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1210 00000100:00100000:1.0:1788572545.232611:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35930:x1875453360455936:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 55838us (56077us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.232620:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4407 00000100:00100000:1.0:1788572545.232995:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382372608:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572545.233045:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572545.233080:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788572545.233086:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382372480:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572545.233108:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572545.233170:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372608 00000100:00100000:0.0:1788572545.233176:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372608 00000100:00100000:0.0:1788572545.233209:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372480 00000100:00100000:0.0:1788572545.233213:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372480 00000100:00100000:0.0:1788572545.233227:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1211 00000100:00100000:0.0:1788572545.233233:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382372608:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572545.233369:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382372608:12345-0@lo:2:osp-syn-1-0.0 Request processed in 135us (323us total) trans 4294967621 rc 0/0 00000100:00100000:0.0:1788572545.233376:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1211 00000100:00100000:0.0:1788572545.233392:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1212 00000100:00100000:0.0:1788572545.233398:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382372480:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572545.233501:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382372480:12345-0@lo:2:osp-syn-1-0.0 Request processed in 103us (391us total) trans 4294967622 rc 0/0 00000100:00100000:0.0:1788572545.233508:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1212 00000004:00080000:1.0:1788572545.233664:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c121f100/2, rc 0, transno 4294967622 00000100:00100000:1.0:1788572545.233672:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382372480:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1788572545.233681:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c121ed80/2, rc 0, transno 4294967621 00000100:00100000:1.0:1788572545.233683:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382372608:0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1788572545.233851:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572545.233855:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb mtime from disk: 0 -> 1788572544 00002000:00010000:2.0:1788572545.233858:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1788572545.233861:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb ctime from disk: 1788572544 -> 1788572544 00002000:00010000:2.0:1788572545.233864:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1788572545.233896:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382372224:12345-0@lo:2:osp-syn-0-0.0 Request processed in 1706us (2259us total) trans 4294967626 rc 0/0 00000100:00100000:2.0:1788572545.233908:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1209 00000100:00100000:2.0:1788572545.234107:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382372864:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572545.234136:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.234155:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572545.234160:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382372736:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572545.234179:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1788572545.234233:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ffc0ca80/2, rc 0, transno 4294967625 00000100:00100000:2.0:1788572545.234240:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382372352:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572545.234248:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ffc0fb80/2, rc 0, transno 4294967626 00000100:00100000:2.0:1788572545.234251:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382372224:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572545.234766:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372864 00000100:00100000:1.0:1788572545.234774:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372864 00000100:00100000:1.0:1788572545.234815:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382372736 00000100:00100000:1.0:1788572545.234819:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382372736 00000100:00100000:1.0:1788572545.234834:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1213 00000100:00100000:1.0:1788572545.234842:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382372864:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1788572545.234991:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788572545.234994:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb mtime from disk: 0 -> 0 00002000:00010000:1.0:1788572545.234996:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572545.234998:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb ctime from disk: 1788572544 -> 1788572544 00002000:00010000:1.0:1788572545.235001:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:1.0:1788572545.235074:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382372864:12345-0@lo:2:osp-syn-1-0.0 Request processed in 232us (936us total) trans 4294967623 rc 0/0 00000100:00100000:1.0:1788572545.235081:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1213 00000100:00100000:1.0:1788572545.235096:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1214 00000100:00100000:1.0:1788572545.235107:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382372736:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1788572545.235171:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:1.0:1788572545.235174:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb mtime from disk: 1788572516 -> 1788572516 00002000:00010000:1.0:1788572545.235177:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb atime from disk: 1788572516 -> 1788572516 00002000:00010000:1.0:1788572545.235179:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb ctime from disk: 1788572527 -> 1788572527 00002000:00010000:1.0:1788572545.235181:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4f:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:1.0:1788572545.235218:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382372736:12345-0@lo:2:osp-syn-1-0.0 Request processed in 115us (1037us total) trans 4294967624 rc 0/0 00000100:00100000:1.0:1788572545.235225:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1214 00000100:00100000:1.0:1788572545.238316:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.238421:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.238455:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457984 00000100:00100000:1.0:1788572545.238464:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457984 00000100:00100000:1.0:1788572545.238523:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1215 00000100:00100000:1.0:1788572545.238534:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360457984:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000004:00080000:2.0:1788572545.239907:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c121fb80/2, rc 0, transno 4294967624 00000100:00100000:2.0:1788572545.239931:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382372736:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1788572545.239944:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c121d880/2, rc 0, transno 4294967623 00000100:00100000:2.0:1788572545.239947:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382372864:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572545.243831:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000004:00080000:3.0:1788572545.244239:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ffc0e680/2, rc 0, transno 4294967624 00000100:00100000:3.0:1788572545.244251:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382372096:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1788572545.244263:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9626c1b59c00/2, rc 0, transno 4294967623 00000100:00100000:3.0:1788572545.244267:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382371968:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572545.244725:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.244770:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360458240 00000100:00100000:2.0:1788572545.244779:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360458240 00000100:00100000:2.0:1788572545.244834:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4413 00000100:00100000:2.0:1788572545.244857:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35932:x1875453360458240:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572545.244977:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35932:x1875453360458240:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 133us (1157us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.244986:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4413 00000100:00100000:3.0:1788572545.245488:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360458368 00000100:00100000:3.0:1788572545.245494:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360458368 00010000:00010000:3.0:1788572545.245524:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 19 pid: 10416 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.245557:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1044 00000100:00100000:3.0:1788572545.245566:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360458368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572545.245578:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572545.245604:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 19 pid: 10416 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.245618:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 19 pid: 10416 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.245645:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.245650:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.245660:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 19 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.245670:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 19 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.245711:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 19 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.245727:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d89c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382372992:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572545.245787:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.246430:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360457856 00000100:00100000:0.0:1788572545.246438:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360457856 00000100:00100000:0.0:1788572545.246500:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 934 00000100:00100000:0.0:1788572545.246508:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+170:35867:x1875453360457856:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572545.246879:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360457856:12345-192.168.202.40@tcp:35:diff.0 Request processed in 371us (8575us total) trans 4294970596 rc 0/0 00000100:00100000:0.0:1788572545.246887:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 934 00010000:00010000:0.0:1788572545.246943:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb1636e0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff10 expref: 19 pid: 10416 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.246961:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382371840:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572545.246988:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 18 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.247001:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2badc00, rc 0) 00000100:00100000:0.0:1788572545.247068:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:35931:x1875453360457472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 18268us (18690us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.247077:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1206 00000100:00100000:1.0:1788572545.262375:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.262427:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.262461:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.262487:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360458752 00000100:00100000:1.0:1788572545.262496:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360458752 00000100:00100000:1.0:1788572545.262561:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1216 00000100:00100000:1.0:1788572545.262574:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35932:x1875453360458752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.262592:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.262611:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff33 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.262648:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.262659:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.262664:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.262670:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.262703:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.262718:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.262734:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382373120:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.262809:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0776c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572545.262892:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360458496 00000100:00100000:1.0:1788572545.262897:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360458496 00000100:00100000:1.0:1788572545.262943:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4414 00000100:00100000:1.0:1788572545.262953:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+171:35867:x1875453360458496:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572545.262966:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.262980:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16371f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff2c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.263066:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.263080:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.263092:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.263107:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.263115:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.263121:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.264172:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360458624 00000100:00100000:0.0:1788572545.264182:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360458624 00002000:00100000:0.0:1788572545.265840:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x52:0x0] (0->2) req@ffff9626c2736a00 x1875453360458624/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572545.265856:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2736a00 with x1875453360458624 ext(0->2) 00002000:00010000:0.0:1788572545.265862:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2736a00 00000100:00100000:0.0:1788572545.265907:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 272 00000100:00100000:0.0:1788572545.268316:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2347:x1875453360458624:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.269311:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d89c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382372992:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572545.269335:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.269488:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+20:2347:x1875453360458624:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 3570us (7058us total) trans 4294967627 rc 0/0 00000100:00100000:0.0:1788572545.269500:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 272 00002000:00100000:0.0:1788572545.269513:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x52:0x0] (0->2) req@ffff9626c2736a00 x1875453360458624/t4294967627(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/448 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572545.269524:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2736a00 with x1875453360458624 ext(0->2) 00002000:00010000:0.0:1788572545.269530:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2736a00 00010000:00010000:1.0:1788572545.276521:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163734 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276543:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276556:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572545.276603:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb3:0x0]: returning size 3 00000004:00200000:1.0:1788572545.276608:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572545.276630:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276647:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276656:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276663:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276670:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276676:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.276680:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276688:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276702:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572545.276705:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572545.276710:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276716:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163726 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276729:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276735:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276741:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276746:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.276749:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276756:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276764:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xbd5240a3 (ffff9627c5716d00) 00010000:00010000:1.0:1788572545.276766:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572545.276770:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276776:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb16372d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276823:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16371f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff2c expref: 173 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276837:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97000/0x85ace67feb163734 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff2c expref: 172 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.276845:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627c5716900) 00010000:00010000:1.0:1788572545.276847:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572545.276849:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97000, rc 0) 00000100:00100000:1.0:1788572545.277089:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:35867:x1875453360458496:12345-192.168.202.40@tcp:101:diff.0 Request processed in 14135us (14722us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.277105:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4414 00000100:00100000:1.0:1788572545.277619:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.277894:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382373120:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572545.277944:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.277963:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572545.278000:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35932:x1875453360458752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 15430us (15537us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.278034:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1216 00000100:00100000:2.0:1788572545.278263:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360458880 00000100:00100000:2.0:1788572545.278271:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360458880 00010000:00010000:2.0:1788572545.278301:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.278330:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1045 00000100:00100000:2.0:1788572545.278339:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360458880:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.278352:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.278372:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.278385:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.278403:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb16370a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff25 expref: 14 pid: 13272 timeout: 993 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.278418:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572545.278422:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572545.278430:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.278440:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.278489:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.278511:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5968700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382373248:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572545.278571:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572545.280721:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360458368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 35141us (35984us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.280737:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1044 00000100:00100000:1.0:1788572545.281619:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.281892:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459008 00000100:00100000:3.0:1788572545.281902:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459008 00000100:00100000:3.0:1788572545.281966:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4415 00000100:00100000:3.0:1788572545.281976:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35931:x1875453360459008:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.282126:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35931:x1875453360459008:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 151us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.282134:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4415 00000100:00100000:1.0:1788572545.291760:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.292066:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459136 00000100:00100000:3.0:1788572545.292077:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459136 00000100:00100000:3.0:1788572545.292139:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1217 00000100:00100000:3.0:1788572545.292150:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2349:x1875453360459136:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572545.292168:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.292189:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb16373b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff3a expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572545.292228:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4e:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572545.292248:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=893s v=(0 0) 00010000:00010000:3.0:1788572545.292253:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.292263:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb16373b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff3a expref: 50 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.292281:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fdf7934b/0x85ace67feb16373b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff3a expref: 50 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.292295:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678200, rc 0) 00000100:00100000:3.0:1788572545.292354:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2349:x1875453360459136:12345-192.168.202.40@tcp:101:diff.0 Request processed in 205us (604us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.292361:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1217 00000100:00100000:1.0:1788572545.292918:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.292995:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459264 00000100:00100000:0.0:1788572545.293021:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459264 00002000:00100000:0.0:1788572545.293064:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x51:0x0] (0->2) req@ffff9627fb64a300 x1875453360459264/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572545.293076:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb64a300 with x1875453360459264 ext(0->2) 00002000:00010000:0.0:1788572545.293081:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb64a300 00000100:00100000:0.0:1788572545.293111:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 273 00000100:00100000:0.0:1788572545.293120:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360459264:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.293565:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360459264:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 444us (647us total) trans 4294967625 rc 0/0 00000100:00100000:0.0:1788572545.293573:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 273 00002000:00100000:0.0:1788572545.293581:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x51:0x0] (0->2) req@ffff9627fb64a300 x1875453360459264/t4294967625(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/448 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572545.293590:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb64a300 with x1875453360459264 ext(0->2) 00002000:00010000:0.0:1788572545.293593:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb64a300 00000100:00100000:2.0:1788572545.294729:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5968700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382373248:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572545.294769:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.294829:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360458880:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 16489us (17210us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.294840:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1045 00000100:00100000:1.0:1788572545.299345:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.299472:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459392 00000100:00100000:0.0:1788572545.299482:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459392 00000100:00100000:0.0:1788572545.299545:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1218 00000100:00100000:0.0:1788572545.299555:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2348:x1875453360459392:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.302303:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.302463:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459520 00000100:00100000:0.0:1788572545.302473:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459520 00000100:00100000:0.0:1788572545.302535:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4416 00000100:00100000:0.0:1788572545.302545:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35932:x1875453360459520:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.302673:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35932:x1875453360459520:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 128us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.302681:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4416 00000100:00100000:3.0:1788572545.305688:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.305730:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.308610:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459776 00000100:00100000:1.0:1788572545.308625:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459776 00000100:00100000:1.0:1788572545.308697:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4417 00000100:00100000:1.0:1788572545.308709:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+172:35867:x1875453360459776:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572545.308731:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.308760:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163742 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff41 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.308876:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163749 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.308893:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.308906:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572545.309715:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb3:0x0]: returning size 3 00010000:00010000:1.0:1788572545.309867:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163742 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff41 expref: 175 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.309893:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163749 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff41 expref: 174 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.309903:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627c5716900) 00010000:00010000:1.0:1788572545.309907:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572545.309910:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96a00, rc 0) 00000100:00100000:1.0:1788572545.311508:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:35867:x1875453360459776:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2792us (5819us total) trans 4294970597 rc 0/0 00000100:00100000:1.0:1788572545.311522:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4417 00000100:00100000:1.0:1788572545.311564:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360459904 00000100:00100000:1.0:1788572545.311573:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360459904 00000100:00100000:1.0:1788572545.311638:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1219 00000100:00100000:1.0:1788572545.311650:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360459904:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.319877:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.320421:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360460160 00000100:00100000:2.0:1788572545.320433:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360460160 00000100:00100000:2.0:1788572545.320511:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 311 00000100:00100000:2.0:1788572545.320523:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9db050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:2348:x1875453360460160:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.320545:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.320577:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163749 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff41 expref: 174 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.320599:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffff9627c5716900) 00010000:00010000:2.0:1788572545.320604:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.320668:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9db050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:2348:x1875453360460160:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 146us (803us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.320676:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 311 00000100:00100000:1.0:1788572545.322669:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.322791:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360460032 00000100:00100000:2.0:1788572545.322801:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360460032 00002000:00100000:2.0:1788572545.322844:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4e:0x0] (0->4095) req@ffff9627fb64b480 x1875453360460032/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.322873:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb64b480 with x1875453360460032 ext(0->4095) 00002000:00010000:2.0:1788572545.322879:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb64b480 00000100:00100000:2.0:1788572545.322905:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 274 00000100:00100000:2.0:1788572545.322915:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2349:x1875453360460032:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572545.324493:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2349:x1875453360460032:12345-192.168.202.40@tcp:3:diff.0 Request processed in 1576us (1831us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572545.324504:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 274 00002000:00100000:2.0:1788572545.324513:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4e:0x0] (0->4095) req@ffff9627fb64b480 x1875453360460032/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.324522:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb64b480 with x1875453360460032 ext(0->4095) 00002000:00010000:2.0:1788572545.324527:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb64b480 00000100:00100000:1.0:1788572545.345699:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.347455:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360460416 00000100:00100000:3.0:1788572545.347471:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360460416 00000100:00100000:3.0:1788572545.347548:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 935 00000100:00100000:3.0:1788572545.347561:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+173:35867:x1875453360460416:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572545.351646:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2348:x1875453360457984:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 113104us (113214us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.351660:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1215 00000001:00080000:2.0:1788572545.351701:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967625 is committed 00000001:00080000:2.0:1788572545.351708:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967622 is committed 00000001:00080000:2.0:1788572545.351719:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967626 is committed 00000100:00100000:2.0:1788572545.352028:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:35867:x1875453360460416:12345-192.168.202.40@tcp:35:diff.0 Request processed in 4444us (6314us total) trans 4294970598 rc 0/0 00000100:00100000:2.0:1788572545.352038:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 935 00000100:00100000:1.0:1788572545.361570:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.361662:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.361694:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360460672 00000100:00100000:1.0:1788572545.361706:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360460672 00000100:00100000:1.0:1788572545.361787:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360460800 00000100:00100000:1.0:1788572545.361791:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360460800 00000100:00100000:1.0:1788572545.361829:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4418 00000100:00100000:1.0:1788572545.361841:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35930:x1875453360460672:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.361930:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572545.361956:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163677 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeca expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.361976:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:1.0:1788572545.361982:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572545.362488:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.362506:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.362517:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572545.362680:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000093c1bc17 reply state 0000000038217ed3 transno 4294970599 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.362690:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572545.362973:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4419 00000100:00100000:0.0:1788572545.362988:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:35867:x1875453360460800:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572545.363027:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.363042:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16375e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff48 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363096:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363107:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363116:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363131:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363138:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363145:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363627:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb163773 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363642:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb163773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363649:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb163773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.363675:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb2:0x0]: returning size 3 00000004:00200000:0.0:1788572545.363679:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572545.363696:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb163773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363708:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb163773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363716:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363723:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363730:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363736:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.363740:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363747:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363758:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:0.0:1788572545.363761:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.363766:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363772:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163765 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363789:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363795:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363977:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363984:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.363987:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.363993:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.364087:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x5abf9a3 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572545.364091:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.364095:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.364102:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb16376c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.364142:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16375e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff48 expref: 176 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.364163:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031600/0x85ace67feb163773 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff48 expref: 175 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.364170:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627fb1e7f00) 00010000:00010000:0.0:1788572545.364173:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572545.364176:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031600, rc 0) 00000100:00100000:1.0:1788572545.364312:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35930:x1875453360460672:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2472us (2754us total) trans 4294970599 rc 0/0 00000100:00100000:1.0:1788572545.364322:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4418 00000100:00100000:0.0:1788572545.365581:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360460800:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2592us (3913us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.365591:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4419 00010000:00010000:3.0:1788572545.376710:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.376728:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.376735:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.376741:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.376747:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.376756:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.376773:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:3.0:1788572545.376777:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.376782:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.376789:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163757 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.383239:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.383981:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360460928 00000100:00100000:1.0:1788572545.383992:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360460928 00000100:00100000:1.0:1788572545.384091:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1220 00000100:00100000:1.0:1788572545.384107:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360460928:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572545.385296:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:1.0:1788572545.385302:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb mtime from disk: 1788572544 -> 1788572522 00002000:00010000:1.0:1788572545.385305:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb atime from disk: 0 -> 1788572522 00002000:00010000:1.0:1788572545.385308:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb ctime from disk: 1788572544 -> 1788572523 00002000:00010000:1.0:1788572545.385310:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x51:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572545.385384:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360460928:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 1283us (2153us total) trans 4294967628 rc 0/0 00000100:00100000:1.0:1788572545.385395:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1220 00000100:00100000:1.0:1788572545.391594:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.392725:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360461184 00000100:00100000:2.0:1788572545.392740:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360461184 00000100:00100000:2.0:1788572545.394316:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 320 00000100:00100000:2.0:1788572545.394353:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:2348:x1875453360461184:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572545.394392:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.394413:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16377a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff4f expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.395678:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.395700:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.395713:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.395729:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.395737:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.395744:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.395942:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.395985:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.396334:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360461056 00000100:00100000:3.0:1788572545.396343:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360461056 00000100:00100000:3.0:1788572545.396399:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1221 00000100:00100000:3.0:1788572545.396408:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2350:x1875453360461056:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572545.396423:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.396438:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb16378f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff64 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572545.396478:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x50:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572545.396496:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=893s v=(0 0) 00010000:00010000:3.0:1788572545.396500:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.396509:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb16378f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff64 expref: 51 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.396525:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ba4ac092/0x85ace67feb16378f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff64 expref: 51 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.396538:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679400, rc 0) 00000100:00100000:3.0:1788572545.396594:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+51:2350:x1875453360461056:12345-192.168.202.40@tcp:101:diff.0 Request processed in 186us (655us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.396602:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1221 00000100:00100000:1.0:1788572545.396776:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360461440 00000100:00100000:1.0:1788572545.396781:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360461440 00000100:00100000:1.0:1788572545.397742:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4420 00000100:00100000:1.0:1788572545.397758:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35930:x1875453360461440:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.398104:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.398120:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.398128:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.398171:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.398191:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 65 pid: 22536 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.398897:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382373376:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.400109:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360459904:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 88455us (94369us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.400123:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1219 00000001:00080000:1.0:1788572545.400160:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967625 is committed 00000001:00080000:1.0:1788572545.400174:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967622 is committed 00000001:00080000:1.0:1788572545.400191:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967624 is committed 00010000:00010000:2.0:1788572545.400560:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb16379d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400578:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb16379d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400590:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb16379d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.400632:0:5842:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb1:0x0]: returning size 3 00000004:00200000:2.0:1788572545.400638:0:5842:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.400663:0:5842:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb16379d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400674:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb16379d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400683:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400691:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400698:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400703:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.400707:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400718:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400731:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.400734:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.400739:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400745:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb163781 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400761:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400770:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400775:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400779:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.400781:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400786:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.400793:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x4e05b2a3 (ffff9627e7a95900) 00010000:00010000:2.0:1788572545.400795:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1788572545.400946:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3c00 going to sleep for 11 seconds 00010000:00010000:2.0:1788572545.401585:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401597:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb163788 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401653:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16377a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff4f expref: 177 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401667:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05a00/0x85ace67feb16379d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff4f expref: 176 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401675:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:2.0:1788572545.401677:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572545.401681:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05a00, rc 0) 00010000:00010000:2.0:1788572545.401692:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.401703:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1637a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff56 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401748:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401760:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401768:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401780:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401787:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.401793:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402687:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1637b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402700:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1637b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402707:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1637b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.402727:0:5842:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb0:0x0]: returning size 3 00000004:00200000:2.0:1788572545.402734:0:5842:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.402755:0:5842:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1637b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402767:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1637b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402776:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402784:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402790:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402795:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.402886:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402894:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402905:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.402907:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.402911:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402978:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb1637ab lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.402991:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403089:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403095:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403099:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.403103:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403108:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403117:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x965f6ba3 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572545.403120:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.403122:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403127:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1637b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403163:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1637a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff56 expref: 178 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403178:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04c00/0x85ace67feb1637b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff56 expref: 177 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403185:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:2.0:1788572545.403187:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572545.403190:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04c00, rc 0) 00010000:00010000:2.0:1788572545.403267:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.403280:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1637c0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff5d expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403318:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403327:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403334:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403345:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403351:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403356:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403636:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1637d5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403645:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1637d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403652:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1637d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.403668:0:5842:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaf:0x0]: returning size 3 00000004:00200000:2.0:1788572545.403671:0:5842:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.403684:0:5842:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1637d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403693:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1637d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403700:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403706:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403712:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403717:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.403720:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403727:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403735:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.403737:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.403740:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.403746:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1637c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.404456:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.404935:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360461696 00000100:00100000:3.0:1788572545.404945:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360461696 00000100:00100000:0.0:1788572545.405306:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4421 00000100:00100000:0.0:1788572545.405319:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360461696:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572545.405340:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.405356:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1637dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff87 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.405470:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1637e3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.405483:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1637e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.405495:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1637e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.405506:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb2:0x0]: returning size 3 00010000:00010000:0.0:1788572545.405639:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1637dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff87 expref: 182 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.405664:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1637e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff87 expref: 181 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.405672:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627fb1e7f00) 00010000:00010000:0.0:1788572545.405675:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572545.405678:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0030e00, rc 0) 00000100:00100000:0.0:1788572545.405758:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360461696:12345-192.168.202.40@tcp:101:diff.0 Request processed in 441us (1313us total) trans 4294970600 rc 0/0 00000100:00100000:0.0:1788572545.405766:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4421 00010000:00010000:2.0:1788572545.405878:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405893:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405900:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405906:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.405911:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405929:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405942:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xdeb924a3 (ffff9627e7a95200) 00010000:00010000:2.0:1788572545.405944:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.405949:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405955:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1637ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.405988:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1637c0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff5d expref: 181 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406477:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e000/0x85ace67feb1637d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff5d expref: 180 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406489:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:2.0:1788572545.406491:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572545.406494:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e000, rc 0) 00010000:00010000:2.0:1788572545.406506:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.406518:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1637f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff6b expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406561:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406571:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406580:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406594:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406601:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406608:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406817:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406824:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406830:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406835:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.406839:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406844:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406852:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.406854:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.406856:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406862:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1637f8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406870:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406874:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406879:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406883:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.406885:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406889:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406894:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xc8545ca3 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.406896:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.406899:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406904:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1637ff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406929:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1637f1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff6b expref: 181 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406938:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.406941:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572545.406944:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1637f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff6b expref: 181 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.406952:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc -2) 00000100:00100000:2.0:1788572545.407035:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:2348:x1875453360461184:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 12706us (15452us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572545.407044:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 320 00000100:00100000:0.0:1788572545.409321:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.409357:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.409382:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462080 00000100:00100000:0.0:1788572545.409390:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462080 00000100:00100000:0.0:1788572545.409436:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4422 00000100:00100000:0.0:1788572545.409465:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35932:x1875453360462080:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.409519:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.409543:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb163703 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff17 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.409562:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572545.409568:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572545.409600:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.409613:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.409629:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572545.409775:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000520ca9eb reply state 000000006f9a330e transno 4294970601 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.409786:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572545.409887:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35932:x1875453360462080:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 442us (530us total) trans 4294970601 rc 0/0 00000100:00100000:0.0:1788572545.409895:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4422 00000100:00100000:0.0:1788572545.410229:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382373376:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572545.410263:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.416103:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360461824 00000100:00100000:0.0:1788572545.416119:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360461824 00010000:00010000:0.0:1788572545.416164:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 67 pid: 22536 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.416212:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1046 00000100:00100000:0.0:1788572545.416222:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360461824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572545.416239:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.416248:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 67 pid: 22536 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.416263:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 67 pid: 22536 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.416292:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163574 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x93:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe76 expref: 67 pid: 22536 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.416309:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572545.416315:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.416325:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.416333:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.416360:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.418386:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360461824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 2156us (9063us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.418400:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1046 00010000:00010000:0.0:1788572545.418530:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418542:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418578:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418585:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418592:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418600:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.418605:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418612:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418628:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572545.418630:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572545.418635:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.418641:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163796 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.419440:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.419547:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.419596:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.419613:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.419629:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.419648:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572545.419672:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.419681:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.419687:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.419692:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.419696:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.419703:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.419715:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:3.0:1788572545.419717:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.419722:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.419728:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163806 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.419763:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462464 00000100:00100000:3.0:1788572545.419770:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462464 00000100:00100000:3.0:1788572545.419823:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462592 00000100:00100000:3.0:1788572545.419826:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462592 00000100:00100000:3.0:1788572545.419839:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462720 00000100:00100000:3.0:1788572545.419842:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462720 00000100:00100000:3.0:1788572545.419865:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462848 00000100:00100000:3.0:1788572545.419868:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462848 00000100:00100000:3.0:1788572545.419881:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1222 00000100:00100000:3.0:1788572545.419890:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360462464:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572545.420564:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572545.420569:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:3.0:1788572545.420572:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:3.0:1788572545.420573:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb ctime from disk: 1788572544 -> 1788572524 00002000:00010000:3.0:1788572545.420575:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x51:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572545.420632:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360462464:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 742us (1035us total) trans 4294967626 rc 0/0 00000100:00100000:3.0:1788572545.420641:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1222 00000100:00100000:3.0:1788572545.420661:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1223 00000100:00100000:3.0:1788572545.420670:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+51:2350:x1875453360462592:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572545.420680:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.420697:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb16380d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff8e expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572545.420713:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4c:0x0] initial LVB size: 3, mtime: 0x6a9b7364, atime: 0x6a9b7364, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572545.420729:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=893s v=(0 0) 00010000:00010000:3.0:1788572545.420734:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.420742:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb16380d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff8e expref: 52 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.420759:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008e368f18/0x85ace67feb16380d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff8e expref: 52 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.420773:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cce00, rc 0) 00000100:00100000:3.0:1788572545.420794:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2350:x1875453360462592:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 127us (1180us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.420815:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1223 00000100:00100000:3.0:1788572545.420830:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1224 00000100:00100000:3.0:1788572545.420837:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+40:2349:x1875453360462720:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572545.420845:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.420856:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095d50bab/0x85ace67feb163814 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff95 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572545.420873:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4e:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736f, blocks: 0xb 00010000:00010000:3.0:1788572545.420882:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=893s v=(0 0) 00010000:00010000:3.0:1788572545.420886:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.420891:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095d50bab/0x85ace67feb163814 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff95 expref: 41 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.420902:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095d50bab/0x85ace67feb163814 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff95 expref: 41 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.420910:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc000, rc 0) 00000100:00100000:3.0:1788572545.420940:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2349:x1875453360462720:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 103us (1310us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.420947:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1224 00000100:00100000:3.0:1788572545.420960:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1225 00000100:00100000:3.0:1788572545.420966:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+40:2350:x1875453360462848:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572545.420973:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.420983:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afd0b343/0x85ace67feb16381b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6eff9c expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572545.420995:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4d:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572545.421241:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=893s v=(0 0) 00010000:00010000:3.0:1788572545.421245:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.421252:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afd0b343/0x85ace67feb16381b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff9c expref: 41 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.421265:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000afd0b343/0x85ace67feb16381b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff9c expref: 41 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.421275:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cdc00, rc 0) 00000100:00100000:3.0:1788572545.421297:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2350:x1875453360462848:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 331us (1647us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.421303:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1225 00000100:00100000:1.0:1788572545.421577:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462208 00000100:00100000:1.0:1788572545.421586:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462208 00002000:00100000:1.0:1788572545.421626:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x50:0x0] (0->4095) req@ffff9626c1d8b100 x1875453360462208/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.421635:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1d8b100 with x1875453360462208 ext(0->4095) 00002000:00010000:1.0:1788572545.421640:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1d8b100 00000100:00100000:1.0:1788572545.421664:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 275 00000100:00100000:1.0:1788572545.421674:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2349:x1875453360462208:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572545.422137:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2349:x1875453360462208:12345-192.168.202.40@tcp:3:diff.0 Request processed in 465us (2701us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572545.422146:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 275 00002000:00100000:1.0:1788572545.422152:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x50:0x0] (0->4095) req@ffff9626c1d8b100 x1875453360462208/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.422162:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1d8b100 with x1875453360462208 ext(0->4095) 00002000:00010000:1.0:1788572545.422166:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1d8b100 00000100:00100000:2.0:1788572545.423967:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360462336 00000100:00100000:2.0:1788572545.423983:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360462336 00000100:00100000:2.0:1788572545.424067:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 312 00000100:00100000:2.0:1788572545.424077:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:2349:x1875453360462336:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.424095:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.424120:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1637e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff87 expref: 180 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.424140:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffff9627fb1e7f00) 00010000:00010000:2.0:1788572545.424145:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.424191:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:2349:x1875453360462336:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 115us (4643us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.424198:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 312 00000100:00100000:0.0:1788572545.424454:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35930:x1875453360461440:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 26691us (28454us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572545.424472:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4420 00000100:00100000:2.0:1788572545.427610:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360459392:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 128045us (128263us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.427624:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1218 00000001:00080000:2.0:1788572545.427659:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967627 is committed 00000100:00100000:2.0:1788572545.442839:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.442887:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.442928:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.442981:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360463104 00000100:00100000:2.0:1788572545.442982:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.442991:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360463104 00000100:00100000:2.0:1788572545.443031:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360463488 00000100:00100000:2.0:1788572545.443035:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360463488 00000100:00100000:0.0:1788572545.443058:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4423 00000100:00100000:1.0:1788572545.443106:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360463744 00000100:00100000:1.0:1788572545.443110:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360463744 00000100:00100000:1.0:1788572545.443155:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 936 00000100:00100000:1.0:1788572545.443165:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd0850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360463744:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572545.443312:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360463232 00000100:00100000:3.0:1788572545.443317:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360463232 00000100:00100000:3.0:1788572545.443348:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1226 00000100:00100000:3.0:1788572545.443355:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360463232:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.443395:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.443397:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35931:x1875453360463104:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.443420:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 12 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.443428:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.443432:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572545.443435:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572545.443436:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.443490:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 12 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.443504:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 12 pid: 6681 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.443518:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382373504:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572545.443593:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb36c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788572545.443622:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb1636d9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff02 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.443644:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:0.0:1788572545.443648:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572545.443679:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.443691:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.443702:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572545.443885:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e0d1bbcb reply state 00000000ed65c13c transno 4294970602 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.443894:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572545.443965:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35931:x1875453360463104:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 899us (1137us total) trans 4294970602 rc 0/0 00000100:00100000:2.0:1788572545.444156:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4424 00000100:00100000:2.0:1788572545.444168:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35932:x1875453360463488:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.444200:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.444211:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.444217:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.444240:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.444250:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 66 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.444261:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5968380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382373632:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572545.444300:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1151780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.447187:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4423 00000100:00100000:1.0:1788572545.447968:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd0850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:35867:x1875453360463744:12345-192.168.202.40@tcp:35:diff.0 Request processed in 4802us (5034us total) trans 4294970603 rc 0/0 00000100:00100000:1.0:1788572545.447980:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 936 00000100:00100000:3.0:1788572545.449214:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382373504:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572545.449243:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.449279:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5968380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382373632:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572545.449296:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450431:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450456:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450464:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450470:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.450475:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450484:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450503:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:1.0:1788572545.450506:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572545.450510:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.450519:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb163829 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.456426:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.456474:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.456505:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.456585:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360463872 00000100:00100000:0.0:1788572545.456594:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360464000 00000100:00100000:3.0:1788572545.456595:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360463872 00000100:00100000:0.0:1788572545.456597:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360464000 00010000:00010000:0.0:1788572545.456637:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 65 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.456637:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 13 pid: 6681 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.456698:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360464128 00000100:00100000:1.0:1788572545.456703:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360464128 00000100:00100000:1.0:1788572545.456753:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1227 00000100:00100000:1.0:1788572545.456765:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360464128:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572545.456987:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572545.456991:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb mtime from disk: 0 -> 1788572522 00002000:00010000:1.0:1788572545.456994:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb atime from disk: 0 -> 1788572522 00002000:00010000:1.0:1788572545.456996:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb ctime from disk: 1788572544 -> 1788572523 00002000:00010000:1.0:1788572545.456998:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x52:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572545.457076:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360464128:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 312us (599us total) trans 4294967629 rc 0/0 00000100:00100000:1.0:1788572545.457084:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1227 00000100:00100000:0.0:1788572545.457221:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1047 00000100:00100000:0.0:1788572545.457230:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360464000:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572545.457242:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.457250:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 65 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457263:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 65 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457290:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16361c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x94:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efeae expref: 65 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457308:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572545.457314:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457324:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457333:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457366:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.457387:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360464000:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 158us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.457394:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1047 00000100:00100000:0.0:1788572545.457408:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1048 00000100:00100000:0.0:1788572545.457415:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360463872:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572545.457423:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.457435:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 14 pid: 6681 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457459:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 14 pid: 6681 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457475:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c6891992/0x85ace67feb1636c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efef4 expref: 14 pid: 6681 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457489:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572545.457494:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572545.457500:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457510:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.457532:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.457550:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360463872:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 136us (1133us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.457557:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1048 00010000:00010000:1.0:1788572545.457606:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457616:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457641:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457649:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457657:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457677:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.457682:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457690:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457700:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:1.0:1788572545.457703:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572545.457707:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.457714:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163830 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.457751:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35932:x1875453360463488:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13587us (14768us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572545.457760:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4424 00000100:00100000:1.0:1788572545.462390:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.462425:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.462462:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360464384 00000100:00100000:1.0:1788572545.462472:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360464384 00000100:00100000:1.0:1788572545.462543:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4425 00000100:00100000:1.0:1788572545.462567:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:35867:x1875453360464384:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572545.462595:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.462616:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163837 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6effa3 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.462736:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16383e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.462753:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16383e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.462765:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16383e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572545.462777:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb1:0x0]: returning size 3 00010000:00010000:1.0:1788572545.462932:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163837 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effa3 expref: 183 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.462951:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16383e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effa3 expref: 182 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.462959:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:1.0:1788572545.462962:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572545.462966:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572545.463049:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360464384:12345-192.168.202.40@tcp:101:diff.0 Request processed in 496us (668us total) trans 4294970604 rc 0/0 00000100:00100000:1.0:1788572545.463056:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4425 00000100:00100000:1.0:1788572545.463083:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360464512 00000100:00100000:1.0:1788572545.463087:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360464512 00000100:00100000:1.0:1788572545.463117:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 321 00000100:00100000:1.0:1788572545.463125:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:2348:x1875453360464512:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572545.463164:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.463174:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16384c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6effaa expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463208:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463218:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463225:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463238:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463245:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463252:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463429:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463436:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463455:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463461:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.463466:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463478:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463488:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572545.463491:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572545.463495:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463501:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163853 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463511:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463516:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463522:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463527:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.463529:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463536:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463543:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x2f385ca3 (ffff9627c5716d00) 00010000:00010000:1.0:1788572545.463546:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572545.463549:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463556:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb16385a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463584:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16384c lrc: 1/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effaa expref: 183 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463595:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572545.463597:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572545.463602:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16384c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effaa expref: 183 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.463610:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc -2) 00000100:00100000:1.0:1788572545.463649:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:2348:x1875453360464512:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 524us (1222us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572545.463656:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 321 00000100:00100000:1.0:1788572545.465650:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.465754:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360464640 00000100:00100000:1.0:1788572545.465762:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360464640 00000100:00100000:1.0:1788572545.465828:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1228 00000100:00100000:1.0:1788572545.465840:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360464640:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.466110:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.466133:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.466150:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.466154:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572545.466160:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.466194:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.466210:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 11 pid: 10416 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.466226:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382373760:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.466296:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3c00 going to sleep for 11 seconds 00010000:00010000:3.0:1788572545.466814:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466827:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466867:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466877:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466885:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466892:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572545.466897:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466907:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466933:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.466942:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8402913/0x85ace67feb163822 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.467420:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360463232:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 24061us (24528us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.467430:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1226 00000100:00100000:0.0:1788572545.471031:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.471126:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360464896 00000100:00100000:3.0:1788572545.471138:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360464896 00000100:00100000:3.0:1788572545.471210:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4426 00000100:00100000:3.0:1788572545.471226:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35931:x1875453360464896:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.471301:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.471318:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 63 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.471327:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.471361:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 63 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.471373:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 63 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.471389:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382373888:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572545.471474:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb36c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572545.472226:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.472250:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.472252:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382373760:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.472272:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572545.472287:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.472293:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360465280 00000100:00100000:1.0:1788572545.472297:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360465280 00000100:00100000:3.0:1788572545.472308:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360465152 00000100:00100000:3.0:1788572545.472311:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360465152 00002000:00100000:1.0:1788572545.472333:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4c:0x0] (0->4095) req@ffff9627dfae4a80 x1875453360465280/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1788572545.472340:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 313 00002000:00100000:1.0:1788572545.472343:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae4a80 with x1875453360465280 ext(0->4095) 00002000:00010000:1.0:1788572545.472347:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfae4a80 00000100:00100000:3.0:1788572545.472351:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360465024 00000100:00100000:3.0:1788572545.472353:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360465024 00000100:00100000:1.0:1788572545.472365:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 276 00010000:00010000:3.0:1788572545.472367:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 12 pid: 10416 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.472373:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2350:x1875453360465280:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572545.472389:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1049 00000100:00100000:3.0:1788572545.472395:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360465024:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572545.472405:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572545.472426:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 12 pid: 10416 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472439:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 12 pid: 10416 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472472:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163718 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x51:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff33 expref: 12 pid: 10416 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472486:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572545.472490:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572545.472495:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472503:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472522:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572545.472555:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360465024:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 159us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.472563:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1049 00010000:00010000:3.0:1788572545.472585:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472594:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472617:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472626:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472633:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472640:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572545.472645:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472654:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472667:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.472674:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb163861 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.472843:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2350:x1875453360465280:12345-192.168.202.40@tcp:3:diff.0 Request processed in 470us (590us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572545.472850:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 276 00002000:00100000:1.0:1788572545.472858:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4c:0x0] (0->4095) req@ffff9627dfae4a80 x1875453360465280/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.472866:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae4a80 with x1875453360465280 ext(0->4095) 00002000:00010000:1.0:1788572545.472870:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfae4a80 00000100:00100000:0.0:1788572545.474172:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:2347:x1875453360465152:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572545.474191:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.474224:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16383e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effa3 expref: 182 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.474243:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffff9627e7a95500) 00010000:00010000:0.0:1788572545.474248:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.474294:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:2347:x1875453360465152:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1946us (2070us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.474302:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 313 00000100:00100000:3.0:1788572545.478958:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360464640:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 13107us (13406us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.478975:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1228 00000100:00100000:3.0:1788572545.481396:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382373888:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572545.481427:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.508109:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.508196:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.508243:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.508323:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360465664 00000100:00100000:3.0:1788572545.508334:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360465664 00000100:00100000:3.0:1788572545.508401:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 937 00000100:00100000:3.0:1788572545.508411:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360465664:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572545.508466:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360465920 00000100:00100000:0.0:1788572545.508471:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360465920 00000100:00100000:0.0:1788572545.508501:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4427 00000100:00100000:0.0:1788572545.508508:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35930:x1875453360465920:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.508568:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508584:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508595:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508712:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508724:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508730:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508762:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.508774:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 65 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.508788:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382374016:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572545.508788:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360465664:12345-192.168.202.40@tcp:35:diff.0 Request processed in 378us (592us total) trans 4294970605 rc 0/0 00000100:00100000:3.0:1788572545.508795:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 937 00000100:00100000:0.0:1788572545.508838:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec068a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.509065:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360465408 00000100:00100000:2.0:1788572545.509070:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360465408 00010000:00010000:2.0:1788572545.509096:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 66 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.509131:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1050 00000100:00100000:2.0:1788572545.509139:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9d9450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360465408:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.509151:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.509157:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 66 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.509170:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 66 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.509188:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163662 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x95:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efebc expref: 66 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.509202:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:2.0:1788572545.509207:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.509218:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.509223:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.509251:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.509273:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9d9450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360465408:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 136us (1177us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.509280:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1050 00010000:00010000:0.0:1788572545.509748:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509758:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509782:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509789:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509796:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509803:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.509808:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509816:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509827:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:0.0:1788572545.509829:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572545.509833:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.509839:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163868 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.509911:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35931:x1875453360464896:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 38689us (38910us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572545.509921:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4426 00000100:00100000:2.0:1788572545.518163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.518254:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.518311:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.518352:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360466560 00000100:00100000:0.0:1788572545.518362:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360466560 00010000:00010000:0.0:1788572545.518402:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 64 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.518438:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1051 00000100:00100000:0.0:1788572545.518477:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360466560:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572545.518494:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.518500:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 64 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.518514:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 64 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.518537:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572545.518543:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.518563:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.518572:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.518592:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.518639:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360466560:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 165us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.518647:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1051 00000100:00100000:2.0:1788572545.518859:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360466176 00000100:00100000:2.0:1788572545.518864:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360466176 00000100:00100000:2.0:1788572545.518897:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360466432 00000100:00100000:2.0:1788572545.518901:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360466432 00000100:00100000:2.0:1788572545.518931:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4428 00000100:00100000:2.0:1788572545.518939:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35932:x1875453360466176:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.519036:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519049:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519058:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519293:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519309:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519317:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519350:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519365:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 65 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.519379:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382374144:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572545.519454:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb540 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.519546:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4429 00000100:00100000:2.0:1788572545.519623:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360466432:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572545.519637:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.519648:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16388b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6effe9 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519725:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163892 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519735:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163892 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519743:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163892 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.519753:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb0:0x0]: returning size 3 00010000:00010000:2.0:1788572545.519839:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb16388b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effe9 expref: 185 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519857:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163892 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effe9 expref: 184 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.519865:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:2.0:1788572545.519868:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572545.519871:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:2.0:1788572545.519907:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360466432:12345-192.168.202.40@tcp:101:diff.0 Request processed in 287us (1595us total) trans 4294970608 rc 0/0 00000100:00100000:2.0:1788572545.519914:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4429 00010000:00010000:3.0:1788572545.520081:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb1635cf lrc: 0/0,0 mode: --/CR res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efe99 expref: 65 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.520101:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382374016:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572545.520122:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.520132:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.520139:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572545.520247:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970606 00000001:00080000:3.0:1788572545.520371:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970609 00000001:00080000:3.0:1788572545.520412:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970610 00000100:00100000:3.0:1788572545.520499:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382374272:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572545.520546:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572545.520583:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.520984:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382374272 00000100:00100000:2.0:1788572545.520991:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382374272 00000100:00100000:2.0:1788572545.521125:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:2.0:1788572545.521135:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382374272:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572545.521212:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382374272:12345-0@lo:1000:mdt00_004.0 Request processed in 78us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.521219:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1788572545.521299:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967626 gen 2 00000004:00080000:3.0:1788572545.521306:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967618 committed transno 4294967626 00000004:00080000:3.0:1788572545.521327:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c1b59c00, transno 4294967623 00000004:00080000:3.0:1788572545.521337:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ffc0e680, transno 4294967624 00000004:00080000:3.0:1788572545.521340:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ffc0ca80, transno 4294967625 00000004:00080000:3.0:1788572545.521343:0:22536:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ffc0fb80, transno 4294967626 00000100:00100000:3.0:1788572545.521349:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382374272:0@lo:1000:mdt00_004.0 00000001:00080000:3.0:1788572545.521717:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970611 00000100:00100000:3.0:1788572545.521779:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382374400:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572545.521810:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572545.529982:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb36c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.534703:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382374400 00000100:00100000:2.0:1788572545.534718:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382374400 00000100:00100000:2.0:1788572545.534790:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:2.0:1788572545.534801:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382374400:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1788572545.535131:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382374400:12345-0@lo:1000:mdt00_004.0 Request processed in 331us (13319us total) trans 4294967630 rc 0/0 00000100:00100000:2.0:1788572545.535144:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00000004:00080000:2.0:1788572545.535422:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:0.0:1788572545.535786:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382374400:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1788572545.535830:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535842:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535863:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535869:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.535874:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535882:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535900:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627ffc51200) 00010000:00010000:0.0:1788572545.535903:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572545.535909:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535916:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163876 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x93:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535957:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535964:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535971:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535977:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.535979:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535985:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.535994:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627d1866400) 00010000:00010000:0.0:1788572545.535997:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572545.536000:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536027:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb15f984 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536048:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536060:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536070:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536078:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536084:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536090:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536096:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.536099:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536106:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536113:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627d1866400) 00010000:00010000:0.0:1788572545.536115:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.536118:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536124:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb1638a0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536137:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536144:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536150:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536155:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.536158:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536164:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536176:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627d1866400) 00010000:00010000:0.0:1788572545.536178:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572545.536181:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.536188:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16386f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.536352:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35930:x1875453360465920:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 27841us (28105us total) trans 4294970612 rc 0/0 00000100:00100000:0.0:1788572545.536362:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4427 00000020:00080000:0.0:1788572545.536366:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:2.0:1788572545.536877:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.536987:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.537057:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.537082:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360466688 00000100:00100000:2.0:1788572545.537088:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360466688 00000100:00100000:2.0:1788572545.537126:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 324 00000100:00100000:2.0:1788572545.537135:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:2348:x1875453360466688:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:2.0:1788572545.537227:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382374144:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572545.537253:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.537507:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360466816 00000100:00100000:1.0:1788572545.537512:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360466816 00000100:00100000:1.0:1788572545.537607:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1229 00000100:00100000:1.0:1788572545.537618:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7660c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360466816:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.537651:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.537670:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.537679:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.537684:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572545.537690:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.537724:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.537737:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 12 pid: 13272 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.537753:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb64bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382374528:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.537815:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.538159:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360466944 00000100:00100000:0.0:1788572545.538164:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360466944 00010000:00010000:0.0:1788572545.538189:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 68 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.538219:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1052 00000100:00100000:0.0:1788572545.538226:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2348:x1875453360466944:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572545.538238:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.538244:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 68 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.538257:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 68 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.538275:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163669 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efec3 expref: 68 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.538292:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572545.538297:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.538307:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.538315:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.538332:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.538360:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2348:x1875453360466944:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 134us (1302us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.538367:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1052 00010000:00010000:2.0:1788572545.538700:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.538710:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572545.538931:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970607 00000001:00080000:2.0:1788572545.539127:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970613 00000001:00080000:2.0:1788572545.539179:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970614 00000100:00100000:2.0:1788572545.539292:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382374656:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572545.539343:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.540072:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572545.540511:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382374656 00000100:00100000:1.0:1788572545.540521:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382374656 00000100:00100000:1.0:1788572545.540594:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:1.0:1788572545.540606:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382374656:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572545.540701:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382374656:12345-0@lo:1000:mdt00_000.0 Request processed in 99us (1357us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.540710:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:2.0:1788572545.540869:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967624 gen 2 00000004:00080000:2.0:1788572545.540875:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967618 committed transno 4294967624 00000004:00080000:2.0:1788572545.540894:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967619 committed transno 4294967624 00000004:00080000:2.0:1788572545.540905:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c121ed80, transno 4294967621 00000004:00080000:2.0:1788572545.540928:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c121f100, transno 4294967622 00000004:00080000:2.0:1788572545.540933:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c121d880, transno 4294967623 00000004:00080000:2.0:1788572545.540936:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c121fb80, transno 4294967624 00000100:00100000:2.0:1788572545.540942:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382374656:0@lo:1000:mdt00_000.0 00000001:00080000:2.0:1788572545.541344:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970615 00000100:00100000:2.0:1788572545.541416:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382374784:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572545.541462:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572545.541486:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628008d8900 going to sleep for 11 seconds 00000004:00080000:0.0:1788572545.541756:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:1.0:1788572545.541987:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382374784 00000100:00100000:1.0:1788572545.541993:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382374784 00000100:00100000:1.0:1788572545.542063:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1788572545.542074:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382374784:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572545.542242:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382374784:12345-0@lo:1000:mdt00_000.0 Request processed in 171us (779us total) trans 4294967627 rc 0/0 00000100:00100000:1.0:1788572545.542250:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:2.0:1788572545.542410:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382374784:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1788572545.542456:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.542467:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.542475:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.542481:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.542486:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.542494:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.542510:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627e7451700) 00010000:00010000:2.0:1788572545.542513:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572545.542519:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.542526:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163884 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x94:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543756:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543769:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543776:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543782:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.543786:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543793:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543805:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:2.0:1788572545.543808:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572545.543812:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543819:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb15faa3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x73:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543846:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 3/0,1 mode: --/EX res: [0x200000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543857:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543867:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543874:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543880:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543887:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543892:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.543895:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543901:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543909:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:2.0:1788572545.543923:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.543926:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 2/0,0 mode: --/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543933:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638ae lrc: 0/0,0 mode: --/EX res: [0x200000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543944:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543951:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543957:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543961:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.543964:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543969:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543975:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:2.0:1788572545.543977:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572545.543980:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.543986:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16387d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x73:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.544250:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35932:x1875453360466176:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 25309us (26093us total) trans 4294970616 rc 0/0 00000100:00100000:2.0:1788572545.544261:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4428 00000020:00080000:2.0:1788572545.544266:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572545.551273:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.551319:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.551405:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360467200 00000100:00100000:3.0:1788572545.551416:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360467200 00000100:00100000:3.0:1788572545.551495:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 314 00000100:00100000:3.0:1788572545.551506:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:2350:x1875453360467200:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572545.551527:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572545.551566:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163892 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6effe9 expref: 186 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.551589:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffff9627e7a95000) 00010000:00010000:3.0:1788572545.551596:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572545.551666:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:2350:x1875453360467200:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 162us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.551674:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 314 00000100:00100000:2.0:1788572545.552293:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360467072 00000100:00100000:2.0:1788572545.552300:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360467072 00002000:00100000:2.0:1788572545.552338:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4e:0x0] (0->4095) req@ffff9627c9b53b80 x1875453360467072/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.552350:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9b53b80 with x1875453360467072 ext(0->4095) 00002000:00010000:2.0:1788572545.552354:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c9b53b80 00000100:00100000:2.0:1788572545.552381:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 277 00000100:00100000:2.0:1788572545.552390:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2350:x1875453360467072:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572545.552597:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2350:x1875453360467072:12345-192.168.202.40@tcp:3:diff.0 Request processed in 207us (1333us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572545.552605:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 277 00002000:00100000:2.0:1788572545.552611:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4e:0x0] (0->4095) req@ffff9627c9b53b80 x1875453360467072/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572545.552619:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9b53b80 with x1875453360467072 ext(0->4095) 00002000:00010000:2.0:1788572545.552623:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c9b53b80 00000100:00100000:1.0:1788572545.566286:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.566327:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.566495:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360467712 00000100:00100000:3.0:1788572545.566507:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360467712 00000100:00100000:3.0:1788572545.566563:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360467968 00000100:00100000:3.0:1788572545.566567:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360467968 00000100:00100000:3.0:1788572545.566585:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 938 00000100:00100000:3.0:1788572545.566594:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35932:x1875453360467712:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572545.567291:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x94:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572545.567397:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970617 00010000:00010000:3.0:1788572545.567458:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.567475:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.567488:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.567498:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.567505:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.567513:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.567530:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572545.567617:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35932:x1875453360467712:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1021us (1340us total) trans 4294970618 rc 0/0 00000100:00100000:3.0:1788572545.567626:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 938 00000020:00080000:3.0:1788572545.567631:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788572545.567649:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 939 00000100:00100000:3.0:1788572545.567658:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:35867:x1875453360467968:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572545.567822:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:35867:x1875453360467968:12345-192.168.202.40@tcp:35:diff.0 Request processed in 164us (1493us total) trans 4294970619 rc 0/0 00000100:00100000:3.0:1788572545.567830:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 939 00010000:00010000:0.0:1788572545.568184:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572545.568193:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.568202:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572545.568207:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.568216:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.568229:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x94:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:0.0:1788572545.568233:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572545.568239:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.568246:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638b5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x94:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.568267:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572545.568269:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572545.568270:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572545.568733:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.568748:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6efff0 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.568799:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.568810:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.568820:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.568835:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.568843:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.568850:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569042:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569050:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569057:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569064:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.569068:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569075:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569084:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.569087:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.569090:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569097:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638c3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569107:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569113:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569120:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569126:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.569129:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569135:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569142:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x1dd915a3 (ffff9627e7a95700) 00010000:00010000:2.0:1788572545.569145:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.569147:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569154:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569182:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638bc lrc: 1/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efff0 expref: 186 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569192:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.569194:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572545.569198:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efff0 expref: 186 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.569207:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc -2) 00000100:00100000:2.0:1788572545.569263:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:2348:x1875453360466688:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 32128us (32275us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572545.569271:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 324 00000100:00100000:1.0:1788572545.571904:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.572649:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360467456 00000100:00100000:3.0:1788572545.572660:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360467456 00000100:00100000:3.0:1788572545.572723:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 940 00000100:00100000:3.0:1788572545.572734:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35930:x1875453360467456:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572545.573329:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x93:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572545.573371:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970620 00010000:00010000:3.0:1788572545.573418:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.573437:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.573464:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.573474:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.573481:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.573487:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.573505:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572545.574094:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35930:x1875453360467456:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1358us (2195us total) trans 4294970621 rc 0/0 00000100:00100000:3.0:1788572545.574105:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 940 00000020:00080000:3.0:1788572545.574110:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788572545.574180:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572545.574188:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.574200:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572545.574204:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.574213:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.574226:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x93:0x0].0x0 (ffff9627d599a600) 00010000:00010000:1.0:1788572545.574229:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572545.574233:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.574239:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1638d1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x93:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.574259:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572545.574261:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572545.574263:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572545.582690:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.582755:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.582781:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360468352 00000100:00100000:2.0:1788572545.582790:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360468352 00010000:00010000:2.0:1788572545.582830:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 13 pid: 13272 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.582880:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1053 00000100:00100000:2.0:1788572545.582890:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360468352:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.582905:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.582931:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 13 pid: 13272 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.582945:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 13 pid: 13272 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.582971:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572545.582977:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572545.582983:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.582993:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.583041:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.583087:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360468352:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 198us (332us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.583096:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1053 00000100:00100000:3.0:1788572545.583297:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360468224 00000100:00100000:3.0:1788572545.583303:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360468224 00000100:00100000:3.0:1788572545.583352:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4430 00000100:00100000:3.0:1788572545.583360:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360468224:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572545.583384:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.583401:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1638d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f00ad expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.583514:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638df lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.583528:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.583539:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.583654:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaf:0x0]: returning size 3 00010000:00010000:3.0:1788572545.583790:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1638d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f00ad expref: 187 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.583810:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f00ad expref: 186 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.583819:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:3.0:1788572545.583823:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572545.583826:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572545.583876:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:35867:x1875453360468224:12345-192.168.202.40@tcp:101:diff.0 Request processed in 517us (1197us total) trans 4294970622 rc 0/0 00000100:00100000:3.0:1788572545.583883:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4430 00010000:00010000:1.0:1788572545.584199:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029fac10e/0x85ace67feb163711 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eff1e expref: 12 pid: 13272 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.584214:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb64bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382374528:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572545.584250:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584262:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584274:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584302:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584311:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584318:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584325:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572545.584330:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584340:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584353:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.584362:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aff4b75a/0x85ace67feb1638a7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.584419:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7660c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360466816:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 46804us (47544us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.584427:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1229 00000100:00100000:3.0:1788572545.588209:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.588320:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360468480 00000100:00100000:2.0:1788572545.588330:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360468480 00000100:00100000:2.0:1788572545.588387:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 327 00000100:00100000:2.0:1788572545.588398:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:2349:x1875453360468480:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:1.0:1788572545.589238:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.589547:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360468608 00000100:00100000:2.0:1788572545.589623:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360468608 00000100:00100000:2.0:1788572545.589685:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 315 00000100:00100000:2.0:1788572545.589694:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:2348:x1875453360468608:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572545.589710:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572545.589734:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1638df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f00ad expref: 188 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.589754:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffff9627e7a95e00) 00010000:00010000:2.0:1788572545.589760:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572545.589805:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+187:2348:x1875453360468608:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 111us (571us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.589812:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 315 00000100:00100000:1.0:1788572545.590457:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.590899:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360468864 00000100:00100000:2.0:1788572545.590908:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360468864 00000100:00100000:2.0:1788572545.590951:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4431 00000100:00100000:2.0:1788572545.590960:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35931:x1875453360468864:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.591083:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591102:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591114:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591361:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591373:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 75 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591383:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591419:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 75 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.591436:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 75 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.591465:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382374912:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572545.591543:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9628008d8900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572545.593124:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.593150:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360468992 00000100:00100000:1.0:1788572545.593156:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360468992 00002000:00100000:1.0:1788572545.593215:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x4d:0x0] (0->4095) req@ffff9627fb1e0e00 x1875453360468992/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.593225:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb1e0e00 with x1875453360468992 ext(0->4095) 00002000:00010000:1.0:1788572545.593231:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb1e0e00 00000100:00100000:1.0:1788572545.593261:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 278 00000100:00100000:1.0:1788572545.593273:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2347:x1875453360468992:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572545.593520:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2347:x1875453360468992:12345-192.168.202.40@tcp:3:diff.0 Request processed in 249us (402us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572545.593528:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 278 00002000:00100000:1.0:1788572545.593536:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x4d:0x0] (0->4095) req@ffff9627fb1e0e00 x1875453360468992/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:206/0 lens 488/4536 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.593544:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb1e0e00 with x1875453360468992 ext(0->4095) 00002000:00010000:1.0:1788572545.593547:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb1e0e00 00010000:00010000:2.0:1788572545.594303:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.594331:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638fb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0067 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.594404:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.594421:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.594434:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.594462:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.594471:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.594478:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595162:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595173:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595180:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595186:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.595191:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595199:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595213:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.595217:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.595222:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595230:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163902 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595243:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595250:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595257:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595262:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.595265:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595272:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595279:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0xa16187a3 (ffff9627e7a95700) 00010000:00010000:2.0:1788572545.595281:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.595284:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595291:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb163909 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595325:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638fb lrc: 1/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0067 expref: 187 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595334:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:2.0:1788572545.595336:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572545.595341:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1638fb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0067 expref: 187 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.595351:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc -2) 00000100:00100000:2.0:1788572545.595428:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:2349:x1875453360468480:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 7031us (7229us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572545.595438:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 327 00000100:00100000:2.0:1788572545.611410:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382374912:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572545.611440:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.615176:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.616751:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360469376 00000100:00100000:0.0:1788572545.616766:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360469376 00000100:00100000:0.0:1788572545.616834:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 941 00000100:00100000:0.0:1788572545.616844:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:35867:x1875453360469376:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572545.617965:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:35867:x1875453360469376:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1116us (2793us total) trans 4294970624 rc 0/0 00000100:00100000:0.0:1788572545.617977:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 941 00000100:00100000:3.0:1788572545.624125:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.625048:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360469120 00000100:00100000:1.0:1788572545.625062:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360469120 00010000:00010000:1.0:1788572545.625132:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 76 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.625308:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1054 00000100:00100000:1.0:1788572545.625323:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2350:x1875453360469120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572545.625338:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572545.625346:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 76 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.625361:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 76 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.625385:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb1636a1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6efee6 expref: 76 pid: 9040 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.625403:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:1.0:1788572545.625409:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.625419:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.625428:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.625497:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572545.627373:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2350:x1875453360469120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 2043us (3247us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.627388:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1054 00010000:00010000:3.0:1788572545.627501:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.627513:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572545.627769:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970623 00000001:00080000:3.0:1788572545.627963:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970625 00000001:00080000:3.0:1788572545.628039:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970626 00000100:00100000:3.0:1788572545.628138:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382375040:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572545.628221:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572545.628261:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda180 going to sleep for 11 seconds 00000100:00100000:3.0:1788572545.628297:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382375040 00000100:00100000:3.0:1788572545.628305:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382375040 00000100:00100000:3.0:1788572545.628347:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:3.0:1788572545.628373:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382375040:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572545.628460:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382375040:12345-0@lo:1000:mdt00_005.0 Request processed in 104us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.628468:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:3.0:1788572545.628511:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382375040:0@lo:1000:mdt00_005.0 00000001:00080000:3.0:1788572545.632739:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970627 00000100:00100000:3.0:1788572545.632869:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382375168:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572545.633168:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572545.633359:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3180 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.635366:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.635622:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360469632 00000100:00100000:1.0:1788572545.635634:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360469632 00000100:00100000:1.0:1788572545.635712:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4432 00000100:00100000:1.0:1788572545.635726:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:35867:x1875453360469632:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572545.635752:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.635776:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163910 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0155 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572545.635896:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00010000:00010000:1.0:1788572545.636286:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163910 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0155 expref: 188 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.636309:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572545.636313:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572545.636320:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163910 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0155 expref: 188 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.636336:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572545.636406:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+187:35867:x1875453360469632:12345-192.168.202.40@tcp:101:diff.0 Request processed in 684us (1049us total) trans 4294970629 rc 301/301 00000100:00100000:1.0:1788572545.636413:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4432 00000100:00100000:0.0:1788572545.637339:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382375168 00000100:00100000:0.0:1788572545.637350:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382375168 00000100:00100000:0.0:1788572545.637397:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 329 00000100:00100000:0.0:1788572545.637406:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382375168:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572545.637689:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:23058:x1875453382375168:12345-0@lo:1000:mdt00_005.0 Request processed in 284us (4522us total) trans 4294967631 rc 0/0 00000100:00100000:0.0:1788572545.637697:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 329 00000100:00100000:1.0:1788572545.639725:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382375168:0@lo:1000:mdt00_005.0 00010000:00010000:1.0:1788572545.639779:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639791:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639805:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639812:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.639816:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639825:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639841:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627ffc51500) 00010000:00010000:1.0:1788572545.639856:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572545.639862:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639870:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb1638f4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x95:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.639906:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.640285:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.640296:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.640303:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.640307:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.640318:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.640329:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627d55e6400) 00010000:00010000:1.0:1788572545.640332:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572545.640336:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.640342:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb15fa09 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640528:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640543:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640565:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640573:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640578:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640583:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640587:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.640591:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640596:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640603:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627d55e6400) 00010000:00010000:2.0:1788572545.640606:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.640609:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640614:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16391e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640624:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640630:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640635:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640640:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.640642:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640648:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640655:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627d55e6400) 00010000:00010000:2.0:1788572545.640657:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572545.640660:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.640666:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1638ed lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.641086:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35931:x1875453360468864:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 50122us (50643us total) trans 4294970628 rc 0/0 00000100:00100000:2.0:1788572545.641098:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4431 00000020:00080000:2.0:1788572545.641104:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:0.0:1788572545.642089:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572545.642096:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572545.642131:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 154 count 101 00000040:00080000:0.0:1788572545.642138:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:0.0:1788572545.642385:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382375296:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572545.642470:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec068540 going to sleep for 11 seconds 00000100:00100000:2.0:1788572545.650716:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.650859:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.650968:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.651537:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470144 00000100:00100000:1.0:1788572545.651547:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470144 00000100:00100000:1.0:1788572545.651631:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 679 00000100:00100000:1.0:1788572545.651643:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efe050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360470144:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572545.651715:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efe050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360470144:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 75us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.651723:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 679 00000100:00100000:2.0:1788572545.652322:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470016 00000100:00100000:2.0:1788572545.652329:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470016 00000100:00100000:2.0:1788572545.652369:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4433 00000100:00100000:2.0:1788572545.652377:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35935:x1875453360470016:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572545.652407:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x72:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572545.652508:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35935:x1875453360470016:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 132us (1539us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.652517:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4433 00000100:00100000:0.0:1788572545.652758:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382375296:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572545.656759:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360469888 00000100:00100000:3.0:1788572545.656778:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360469888 00000100:00100000:3.0:1788572545.656873:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 942 00000100:00100000:3.0:1788572545.656884:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35931:x1875453360469888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572545.658435:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x95:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572545.658494:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970630 00010000:00010000:3.0:1788572545.658546:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.658876:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.658892:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.658902:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.658907:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.658914:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.658943:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572545.659349:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.659394:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.659415:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470528 00000100:00100000:1.0:1788572545.659422:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470528 00000100:00100000:1.0:1788572545.659486:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 943 00000100:00100000:1.0:1788572545.659499:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+187:35867:x1875453360470528:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572545.659699:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360470528:12345-192.168.202.40@tcp:35:diff.0 Request processed in 203us (302us total) trans 4294970632 rc 0/0 00000100:00100000:1.0:1788572545.659709:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 943 00010000:00010000:2.0:1788572545.659945:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572545.659953:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.659961:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572545.659966:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.659980:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.659994:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x95:0x0].0x0 (ffff9627d599a700) 00010000:00010000:2.0:1788572545.659997:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572545.660023:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.660030:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163925 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x95:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.660049:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572545.660051:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572545.660053:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572545.660261:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35931:x1875453360469888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 3377us (9554us total) trans 4294970631 rc 0/0 00000100:00100000:3.0:1788572545.660270:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 942 00000020:00080000:3.0:1788572545.660274:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788572545.661803:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470272 00000100:00100000:1.0:1788572545.661820:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470272 00000100:00100000:1.0:1788572545.661959:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4434 00000100:00100000:1.0:1788572545.661972:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35937:x1875453360470272:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572545.662051:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x75:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572545.662170:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35937:x1875453360470272:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 201us (2824us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.662178:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4434 00000100:00100000:1.0:1788572545.678869:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.679269:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470656 00000100:00100000:0.0:1788572545.679279:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470656 00000100:00100000:0.0:1788572545.679398:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4435 00000100:00100000:0.0:1788572545.679408:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35936:x1875453360470656:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572545.679453:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x70:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572545.680363:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35936:x1875453360470656:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 952us (1513us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.680373:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4435 00000100:00100000:3.0:1788572545.683380:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.690869:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.691362:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470784 00000100:00100000:2.0:1788572545.691374:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470784 00000100:00100000:3.0:1788572545.691764:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.692088:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360470912 00000100:00100000:0.0:1788572545.692094:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360470912 00000100:00100000:0.0:1788572545.692141:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360471040 00000100:00100000:0.0:1788572545.692145:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360471040 00000100:00100000:0.0:1788572545.692172:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4436 00000100:00100000:0.0:1788572545.692181:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35935:x1875453360470912:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.692271:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692289:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692302:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692318:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692331:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692336:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692363:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692371:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692377:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.692416:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x72:0x0]: returning size 3 00000004:00200000:0.0:1788572545.692421:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572545.692435:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692458:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692466:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692473:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692479:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.692484:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692493:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692505:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572545.692508:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.692512:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692518:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb16392c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692534:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692540:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692546:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692559:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.692562:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692568:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692575:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x73188273 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572545.692577:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.692580:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692590:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163933 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x73188273 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692603:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692609:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692614:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692619:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.692622:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692628:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692633:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627d55e6f00) 00010000:00010000:0.0:1788572545.692635:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.692638:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692643:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16393a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.692718:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35935:x1875453360470912:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 536us (1858us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.692726:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4436 00000100:00100000:0.0:1788572545.692746:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4437 00000100:00100000:0.0:1788572545.692754:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35937:x1875453360471040:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.692777:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692786:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692794:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692805:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692812:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692819:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692835:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692842:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692859:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.692879:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x75:0x0]: returning size 3 00000004:00200000:0.0:1788572545.692882:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572545.692888:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692896:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692903:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692909:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.692914:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.692916:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693136:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693145:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572545.693147:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.693150:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693156:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163941 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693172:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693179:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693185:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693192:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.693194:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693200:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693207:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x67a8855 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572545.693210:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.693212:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693219:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163948 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x67a8855 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693230:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693235:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693241:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693247:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.693249:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693255:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693265:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627c29da300) 00010000:00010000:0.0:1788572545.693268:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.693271:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.693276:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16394f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.693311:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35937:x1875453360471040:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 557us (1548us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.693317:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4437 00000100:00100000:3.0:1788572545.693489:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.693622:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360471168 00000100:00100000:2.0:1788572545.693628:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360471168 00000100:00100000:2.0:1788572545.693651:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4438 00000100:00100000:2.0:1788572545.693658:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360470784:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572545.693673:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.693686:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163956 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f015c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.693728:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.693738:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.693750:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.693763:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.693770:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.693777:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694172:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694181:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694187:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.694212:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xac:0x0]: returning size 3 00000004:00200000:2.0:1788572545.694216:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.694232:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694240:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694247:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694257:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694264:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694268:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.694272:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694278:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694286:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572545.694288:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.694292:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694297:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb16395d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694307:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694312:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694318:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694324:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.694327:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694333:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694339:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x9beb5ca3 (ffff9627e7a95700) 00010000:00010000:2.0:1788572545.694341:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.694344:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694349:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163964 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694372:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163956 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 190 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694384:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 189 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694391:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:2.0:1788572545.694393:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572545.694395:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:2.0:1788572545.694425:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+189:35867:x1875453360470784:12345-192.168.202.40@tcp:101:diff.0 Request processed in 767us (11057us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.694431:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4438 00000100:00100000:2.0:1788572545.694464:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4439 00000100:00100000:2.0:1788572545.694474:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35936:x1875453360471168:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.694504:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694513:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694521:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694531:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694538:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694544:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694775:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694782:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694790:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.694811:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x70:0x0]: returning size 3 00000004:00200000:2.0:1788572545.694814:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.694821:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694828:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.694839:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695023:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695030:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.695033:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695039:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695049:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572545.695052:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.695055:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695061:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163972 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695071:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695076:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695082:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695088:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.695091:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695097:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695103:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xc98029 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.695105:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.695108:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695113:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163979 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xc98029 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695123:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695129:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695134:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695139:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.695141:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695147:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695154:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:2.0:1788572545.695156:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.695159:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.695165:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163980 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.695204:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35936:x1875453360471168:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 732us (1715us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.695211:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4439 00000100:00100000:1.0:1788572545.710581:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.710632:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.710662:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360471424 00000100:00100000:1.0:1788572545.710663:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.710676:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360471424 00000100:00100000:2.0:1788572545.710741:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360471680 00000100:00100000:2.0:1788572545.710745:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360471680 00000100:00100000:1.0:1788572545.710763:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.710763:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472064 00000100:00100000:2.0:1788572545.710766:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472064 00000100:00100000:2.0:1788572545.710776:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472192 00000100:00100000:2.0:1788572545.710778:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472192 00000100:00100000:1.0:1788572545.710785:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4440 00000100:00100000:1.0:1788572545.710813:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35935:x1875453360471424:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.710836:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.710862:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163987 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0163 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.710987:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711049:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711064:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572545.711082:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x72:0x0]: returning size 3 00010000:00010000:1.0:1788572545.711192:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711210:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711219:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711226:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.711230:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711238:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711254:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627d55e6f00) 00010000:00010000:1.0:1788572545.711257:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572545.711262:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711269:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16398e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711311:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163987 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0163 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711321:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572545.711323:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572545.711328:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163987 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0163 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.711337:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572545.711390:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35935:x1875453360471424:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 596us (820us total) trans 4294970633 rc 301/301 00000100:00100000:2.0:1788572545.713765:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4441 00000100:00100000:2.0:1788572545.713792:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+189:35867:x1875453360471680:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572545.713815:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.713837:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16399c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f016a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.713944:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.713959:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.713970:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.713987:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xac:0x0]: returning size 3 00010000:00010000:2.0:1788572545.714101:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714110:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714117:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714122:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.714127:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714134:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714145:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:2.0:1788572545.714148:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572545.714152:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714158:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb1639a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714200:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16399c lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f016a expref: 191 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714210:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572545.714213:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572545.714219:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16399c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f016a expref: 191 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714229:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f200, rc 0) 00000100:00100000:2.0:1788572545.714282:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360471680:12345-192.168.202.40@tcp:101:diff.0 Request processed in 504us (3650us total) trans 4294970634 rc 301/301 00000100:00100000:2.0:1788572545.714290:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4441 00000100:00100000:2.0:1788572545.714313:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4442 00000100:00100000:2.0:1788572545.714321:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35936:x1875453360472064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.714330:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.714339:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639b1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0178 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714382:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714391:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714399:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.714407:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x70:0x0]: returning size 3 00010000:00010000:2.0:1788572545.714457:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714466:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714472:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714478:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.714480:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714486:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714495:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:2.0:1788572545.714496:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572545.714500:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714506:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639b8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714529:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639b1 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0178 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714538:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572545.714540:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572545.714543:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639b1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0178 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714552:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:2.0:1788572545.714574:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35936:x1875453360472064:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 254us (3911us total) trans 4294970635 rc 301/301 00000100:00100000:2.0:1788572545.714581:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4442 00000100:00100000:2.0:1788572545.714596:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4443 00000100:00100000:2.0:1788572545.714603:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35937:x1875453360472192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.714612:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.714620:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1639c6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0171 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714661:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714670:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714678:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.714686:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x75:0x0]: returning size 3 00010000:00010000:2.0:1788572545.714722:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714728:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714735:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714740:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.714744:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714751:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714759:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627c29da300) 00010000:00010000:2.0:1788572545.714762:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572545.714765:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714772:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714794:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1639c6 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0171 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714802:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572545.714804:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572545.714808:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1639c6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0171 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.714817:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:2.0:1788572545.714838:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35937:x1875453360472192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 235us (4074us total) trans 4294970636 rc 301/301 00000100:00100000:2.0:1788572545.714845:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4443 00000100:00100000:1.0:1788572545.721232:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4440 00000100:00100000:1.0:1788572545.721336:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.721853:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472320 00000100:00100000:2.0:1788572545.721863:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472320 00000100:00100000:2.0:1788572545.721923:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4444 00000100:00100000:2.0:1788572545.721933:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360472320:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572545.721961:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.721986:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 2/0,0 mode: --/CR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f017f expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.722071:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.722084:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 4/0,0 mode: --/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 191 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.722106:0:9040:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.722118:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 3/0,0 mode: --/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 191 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.722132:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:2.0:1788572545.722136:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572545.722140:0:9040:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 3/0,0 mode: --/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 191 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.722149:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fe00, rc 0) 00000100:00100000:2.0:1788572545.722207:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:35867:x1875453360472320:12345-192.168.202.40@tcp:101:diff.0 Request processed in 275us (871us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.722214:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4444 00000100:00100000:1.0:1788572545.726827:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.726882:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472448 00000100:00100000:3.0:1788572545.726892:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472448 00000100:00100000:3.0:1788572545.726956:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4445 00000100:00100000:3.0:1788572545.726966:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35937:x1875453360472448:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572545.727045:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x75:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572545.727168:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35937:x1875453360472448:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 201us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.727177:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4445 00000100:00100000:0.0:1788572545.736626:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.736722:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472576 00000100:00100000:2.0:1788572545.736733:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472576 00000100:00100000:2.0:1788572545.736800:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4446 00000100:00100000:2.0:1788572545.736811:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35935:x1875453360472576:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572545.736856:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x72:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572545.736977:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35935:x1875453360472576:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 168us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.736985:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4446 00000100:00100000:1.0:1788572545.741507:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.741577:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.741599:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472832 00000100:00100000:2.0:1788572545.741609:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472832 00000100:00100000:2.0:1788572545.741670:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360472704 00000100:00100000:2.0:1788572545.741674:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360472704 00000100:00100000:2.0:1788572545.741706:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4447 00000100:00100000:2.0:1788572545.741716:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35937:x1875453360472832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.741726:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4448 00000100:00100000:0.0:1788572545.741735:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35936:x1875453360472704:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.741739:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000004:00000080:0.0:1788572545.741758:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x70:0x0] from 18446744073709551615 #0 00010000:00010000:2.0:1788572545.741765:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1639e2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0186 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.741831:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.741857:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.741871:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.741882:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35936:x1875453360472704:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 148us (305us total) trans 0 rc 0/0 00010000:00010000:2.0:1788572545.741888:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.741890:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4448 00010000:00010000:2.0:1788572545.741895:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.741901:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742242:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742253:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742262:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.742288:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:2.0:1788572545.742292:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.742314:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742325:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742331:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742338:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742343:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.742348:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742357:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742369:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572545.742372:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.742378:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742384:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1639e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742401:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742406:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742413:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742418:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.742421:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742427:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742434:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.742437:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.742440:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742465:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1639f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742501:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1639e2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0186 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742516:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0186 expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.742530:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572545.742532:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572545.742535:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc 0) 00000100:00100000:2.0:1788572545.742580:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35937:x1875453360472832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 866us (1086us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.742587:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4447 00000100:00100000:1.0:1788572545.754966:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.755043:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.755194:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360473088 00000100:00100000:3.0:1788572545.755206:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360473088 00000100:00100000:3.0:1788572545.755267:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360473216 00000100:00100000:3.0:1788572545.755271:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360473216 00000100:00100000:3.0:1788572545.755298:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4449 00000100:00100000:3.0:1788572545.755307:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35937:x1875453360473088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.755329:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.755333:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572545.755360:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1639f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0186 expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.755380:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.755386:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572545.755403:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1639fe lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f018d expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572545.755954:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x52:0x0], index 1: 0 00010000:00010000:3.0:1788572545.756195:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163a05 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.756211:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163a05 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.756221:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163a05 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.756288:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1639fe lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f018d expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.756303:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163a05 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f018d expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.756312:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627d599a300) 00010000:00010000:3.0:1788572545.756314:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572545.756317:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc800, rc 0) 00000100:00100000:3.0:1788572545.756391:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35937:x1875453360473088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1085us (1434us total) trans 4294970637 rc 0/0 00000100:00100000:3.0:1788572545.756400:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4449 00000100:00100000:3.0:1788572545.756424:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4450 00000100:00100000:3.0:1788572545.756433:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35936:x1875453360473216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.756544:0:7938:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b53480 pname:cluuid:pid:xid:nid:opc:job osp-pre-1-0:lustre-MDT0000-mdtlov_UUID:7938:1875453382375424:0@lo:5:osp-pre-1-0.0 00000100:00100000:0.0:1788572545.756590:0:7938:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572545.756646:0:7938:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec068540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.756664:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382375424 00000100:00100000:0.0:1788572545.756669:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382375424 00000100:00100000:0.0:1788572545.756701:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1230 00000100:00100000:0.0:1788572545.756707:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:7938:x1875453382375424:12345-0@lo:5:osp-pre-1-0.0 00002000:00080000:0.0:1788572545.756752:0:20911:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0001: reserve 32 objects in group 0x280000400 at 98 00010000:00010000:3.0:1788572545.757735:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.757754:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163a13 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0194 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.757809:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.757819:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.757829:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.757839:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.757844:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.757849:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758061:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163a28 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758069:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163a28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758074:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163a28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.758095:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:3.0:1788572545.758098:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.758110:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163a28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758118:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758122:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758130:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758134:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.758137:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758142:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758150:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572545.758152:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.758156:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758160:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163a1a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758168:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758171:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758175:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758177:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.758179:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758185:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758189:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627d599a700) 00010000:00010000:3.0:1788572545.758190:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.758192:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758197:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163a21 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758227:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163a13 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0194 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758239:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163a28 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0194 expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.758247:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.758249:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572545.758252:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572545.758302:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35936:x1875453360473216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1870us (3259us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.758309:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4450 00000100:00100000:1.0:1788572545.770783:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.770814:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.770948:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360473600 00000100:00100000:3.0:1788572545.770958:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360473600 00000100:00100000:3.0:1788572545.771033:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360473472 00000100:00100000:3.0:1788572545.771037:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360473472 00000100:00100000:3.0:1788572545.771061:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4451 00000100:00100000:3.0:1788572545.771071:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35937:x1875453360473600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.771096:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.771116:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163a2f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01a2 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771188:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771204:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771216:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771231:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771239:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771246:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771545:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb163a44 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771556:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb163a44 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771564:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb163a44 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.771596:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:3.0:1788572545.771600:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.771618:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb163a44 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771631:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771638:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771646:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771652:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.771657:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771665:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771676:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572545.771680:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.771685:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771692:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163a36 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771705:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771715:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771722:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771727:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.771730:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771736:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771743:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627d599a600) 00010000:00010000:3.0:1788572545.771745:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.771748:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771754:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163a3d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771784:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb163a2f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a2 expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771797:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb163a44 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a2 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.771805:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.771807:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572545.771810:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246c000, rc 0) 00000100:00100000:3.0:1788572545.771872:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35937:x1875453360473600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 801us (1101us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.771881:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4451 00000100:00100000:3.0:1788572545.771998:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4452 00000100:00100000:3.0:1788572545.772032:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35936:x1875453360473472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.772042:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.772045:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572545.772057:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163a28 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0194 expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.772068:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.772072:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572545.772081:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb163a4b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f019b expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572545.772434:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x53:0x0], index 0: 0 00010000:00010000:3.0:1788572545.772639:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb163a52 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.772650:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb163a52 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.772659:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb163a52 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.772707:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb163a4b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f019b expref: 89 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.772726:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb163a52 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f019b expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.772735:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:3.0:1788572545.772738:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572545.772741:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea3800, rc 0) 00000100:00100000:3.0:1788572545.772781:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35936:x1875453360473472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 775us (1966us total) trans 4294970638 rc 0/0 00000100:00100000:3.0:1788572545.772789:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4452 00000100:00100000:3.0:1788572545.779617:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.779761:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360473856 00000100:00100000:2.0:1788572545.779772:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360473856 00000100:00100000:2.0:1788572545.779840:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4453 00000100:00100000:2.0:1788572545.779851:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35936:x1875453360473856:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.779913:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572545.779941:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb163a52 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f019b expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.779963:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627d599aa00) 00010000:00010000:2.0:1788572545.779980:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb163a44 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a2 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.779990:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572545.779994:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572545.780047:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780061:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780076:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780091:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780098:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780106:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780316:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780325:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780332:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780339:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.780344:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780352:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780362:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572545.780365:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.780369:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780377:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163a60 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780387:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780393:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780400:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780405:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572545.780408:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780414:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780421:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x5efe9704 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.780423:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572545.780426:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.780432:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163a67 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x5efe9704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.780533:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35936:x1875453360473856:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 682us (926us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572545.780542:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4453 00000100:00100000:2.0:1788572545.782493:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.782591:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360473984 00000100:00100000:3.0:1788572545.782600:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360473984 00000100:00100000:3.0:1788572545.782655:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4454 00000100:00100000:3.0:1788572545.782665:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35935:x1875453360473984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.782684:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.782704:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163a6e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01a9 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.782776:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.782789:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.782800:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.782836:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.782845:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.782852:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783658:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783671:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783680:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572545.783709:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:3.0:1788572545.783713:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572545.783730:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783743:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783751:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783758:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783770:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.783775:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783783:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783795:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572545.783798:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.783803:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783810:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163a75 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783823:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783829:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783836:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783841:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.783844:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783850:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783862:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9626c1d8fa00) 00010000:00010000:3.0:1788572545.783865:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.783868:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783875:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163a7c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.783912:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163a6e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.784171:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.784179:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.784181:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572545.784183:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e76ea200, rc 0) 00000100:00100000:3.0:1788572545.784259:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35935:x1875453360473984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1594us (1774us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.784267:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4454 00000100:00100000:1.0:1788572545.786163:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.786212:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.786291:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360474112 00000100:00100000:0.0:1788572545.786301:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360474112 00000100:00100000:0.0:1788572545.786356:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360474368 00000100:00100000:0.0:1788572545.786360:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360474368 00000100:00100000:0.0:1788572545.786383:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4455 00000100:00100000:0.0:1788572545.786392:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35936:x1875453360474112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.786412:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.786433:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb163a8a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01b0 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786483:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786496:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786508:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786608:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb163a8a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786628:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786637:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572545.786640:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572545.786644:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0031c00, rc 0) 00000100:00100000:0.0:1788572545.786691:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35936:x1875453360474112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 300us (535us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.786703:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4455 00000100:00100000:0.0:1788572545.786721:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4456 00000100:00100000:0.0:1788572545.786728:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35937:x1875453360474368:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.786757:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572545.786773:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163a05 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f018d expref: 88 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786785:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627d599a300) 00010000:00010000:0.0:1788572545.786794:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572545.786809:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786820:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786830:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786862:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.786877:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 87 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.786892:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b52a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382375552:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572545.792330:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda180 going to sleep for 11 seconds 00000100:00100000:1.0:1788572545.792745:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.792859:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360474624 00000100:00100000:3.0:1788572545.792868:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360474624 00000100:00100000:3.0:1788572545.792945:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4457 00000100:00100000:3.0:1788572545.792956:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35936:x1875453360474624:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.793084:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.793106:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.793118:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572545.793551:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].71, 88 off13960 00000004:00010000:3.0:1788572545.793645:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f0f6b775 reply state 00000000bf5ce409 transno 4294970639 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.793655:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572545.793761:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35936:x1875453360474624:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 805us (1018us total) trans 4294970639 rc 0/0 00000100:00100000:3.0:1788572545.793769:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4457 00000100:00100000:0.0:1788572545.794439:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b52a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382375552:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572545.794479:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.798630:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572545.798698:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.798711:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.798719:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.798726:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e76eba00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.798732:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.798741:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.798762:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572545.798765:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572545.798770:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.798777:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb163a9f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.798853:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360474752 00000100:00100000:1.0:1788572545.798864:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360474752 00010000:00010000:1.0:1788572545.798922:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 89 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.798961:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1055 00000100:00100000:1.0:1788572545.798974:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9d8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:2350:x1875453360474752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572545.798989:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572545.798996:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 89 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.799031:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 89 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.799051:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76ea200/0x85ace67feb163a83 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01a9 expref: 89 pid: 23058 timeout: 993 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.799063:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572545.799068:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.799077:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.799085:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.799111:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572545.799163:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9d8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:2350:x1875453360474752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 193us (543us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.799171:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1055 00010000:00010000:0.0:1788572545.799496:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799509:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799535:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799545:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799553:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799746:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799754:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799761:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799767:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.799771:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799778:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799789:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572545.799792:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.799795:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799803:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c200/0x85ace67feb163a98 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799818:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799824:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799831:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799836:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.799839:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799845:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799851:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x51e4dc80 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572545.799854:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.799857:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.799862:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb163aa6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x51e4dc80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.799965:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35937:x1875453360474368:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 13235us (13750us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572545.799973:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4456 00000100:00100000:0.0:1788572545.800906:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.805356:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:7938:x1875453382375424:12345-0@lo:5:osp-pre-1-0.0 Request processed in 48639us (48755us total) trans 4294967628 rc 0/0 00000100:00100000:2.0:1788572545.805370:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1230 00000100:00100000:1.0:1788572545.806183:0:7938:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b53480 pname:cluuid:pid:xid:nid:opc:job osp-pre-1-0:lustre-MDT0000-mdtlov_UUID:7938:1875453382375424:0@lo:5:osp-pre-1-0.0 00000004:00080000:1.0:1788572545.806212:0:7938:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0001-osc-MDT0000: current precreated pool: [0x280000400:0x52:0x0]-[0x280000400:0x81:0x0] 00000100:00100000:0.0:1788572545.806547:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.806583:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.806612:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360474880 00000100:00100000:0.0:1788572545.806619:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360474880 00000100:00100000:0.0:1788572545.806658:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360475008 00000100:00100000:0.0:1788572545.806662:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360475008 00000100:00100000:0.0:1788572545.806682:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360475264 00000100:00100000:0.0:1788572545.806685:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360475264 00000100:00100000:0.0:1788572545.806731:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4458 00000100:00100000:0.0:1788572545.806741:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35936:x1875453360474880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.806761:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.806785:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 2/0,0 mode: --/CR res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01b7 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.806850:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.806866:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.806887:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572545.806890:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572545.806894:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:0.0:1788572545.806944:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35936:x1875453360474880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 204us (6041us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.806950:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4458 00000100:00100000:0.0:1788572545.807049:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4459 00000100:00100000:0.0:1788572545.807058:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35937:x1875453360475008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.807070:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.807083:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163ab4 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01be expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807108:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807117:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807125:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807282:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb163ab4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807302:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807310:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572545.807313:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572545.807316:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c2477200, rc 0) 00000100:00100000:0.0:1788572545.807343:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35937:x1875453360475008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 286us (799us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.807349:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4459 00000100:00100000:0.0:1788572545.807365:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4460 00000100:00100000:0.0:1788572545.807372:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35935:x1875453360475264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.807380:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.807390:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb163ac2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01c5 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572545.807771:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x53:0x0], index 1: 0 00010000:00010000:0.0:1788572545.807966:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb163ac9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807978:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb163ac9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.807987:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb163ac9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.808065:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb163ac2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01c5 expref: 92 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.808080:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb163ac9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01c5 expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.808088:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627fb1e7300) 00010000:00010000:0.0:1788572545.808090:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572545.808093:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc 0) 00000100:00100000:0.0:1788572545.808131:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35935:x1875453360475264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 759us (1547us total) trans 4294970640 rc 0/0 00000100:00100000:0.0:1788572545.808138:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4460 00000100:00100000:0.0:1788572545.813535:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.813581:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.813620:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.813736:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360475392 00000100:00100000:3.0:1788572545.813748:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360475392 00000100:00100000:3.0:1788572545.813809:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1231 00000100:00100000:3.0:1788572545.813820:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360475392:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.814159:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360475392:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 340us (635us total) trans 4294967632 rc 0/0 00000100:00100000:3.0:1788572545.814168:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1231 00000100:00100000:2.0:1788572545.816795:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360475648 00000100:00100000:2.0:1788572545.816810:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360475648 00000100:00100000:2.0:1788572545.816859:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360475776 00000100:00100000:2.0:1788572545.816862:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360475776 00000100:00100000:2.0:1788572545.817028:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4461 00000100:00100000:2.0:1788572545.817039:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35937:x1875453360475648:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.817142:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.817160:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.817172:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.817562:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4462 00000100:00100000:0.0:1788572545.817572:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35935:x1875453360475776:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.817584:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572545.817596:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163ade lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01cc expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817641:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817651:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817659:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817672:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817678:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817685:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817870:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163af3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817880:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163af3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817886:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163af3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572545.817909:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:0.0:1788572545.817926:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572545.817946:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163af3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817958:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817964:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817971:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817977:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.817981:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.817989:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818000:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572545.818025:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.818030:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818037:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163ae5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818050:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818060:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818066:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818071:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572545.818073:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818080:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818087:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627fb1e7700) 00010000:00010000:0.0:1788572545.818089:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572545.818092:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818097:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163aec lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818128:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163ade lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01cc expref: 94 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818140:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163af3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01cc expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.818147:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572545.818149:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572545.818152:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bce00, rc 0) 00000100:00100000:0.0:1788572545.818217:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35935:x1875453360475776:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 646us (4594us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.818223:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4462 00000040:00080000:2.0:1788572545.818698:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].68, 88 off13768 00000004:00010000:2.0:1788572545.818796:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fba0ac16 reply state 00000000d4378cd6 transno 4294970641 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.818805:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572545.818859:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35937:x1875453360475648:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1821us (5276us total) trans 4294970641 rc 0/0 00000100:00100000:2.0:1788572545.818867:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4461 00000100:00100000:0.0:1788572545.824773:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.824804:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.824882:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.825072:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360475904 00000100:00100000:3.0:1788572545.825088:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360475904 00000100:00100000:3.0:1788572545.825142:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360476160 00000100:00100000:3.0:1788572545.825146:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360476160 00000100:00100000:3.0:1788572545.825171:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4463 00000100:00100000:3.0:1788572545.825181:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35937:x1875453360475904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.825218:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572545.825240:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01d3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825303:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825319:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825340:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:3.0:1788572545.825343:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572545.825347:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d800, rc 0) 00000100:00100000:3.0:1788572545.825401:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35937:x1875453360475904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 221us (637us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.825409:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4463 00000100:00100000:3.0:1788572545.825428:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4464 00000100:00100000:3.0:1788572545.825435:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35935:x1875453360476160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572545.825490:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572545.825509:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb163ac9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01c5 expref: 95 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825525:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627fb1e7300) 00010000:00010000:3.0:1788572545.825541:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163af3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01cc expref: 94 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825554:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.825558:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572545.825574:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825584:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825592:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825605:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825612:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825618:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825814:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825823:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825830:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825835:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.825839:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825847:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825857:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572545.825859:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.825864:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825870:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163b01 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825878:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825883:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825889:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825894:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.825896:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825901:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.825907:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x4874e45a (ffff9626c1d8fa00) 00010000:00010000:3.0:1788572545.825910:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572545.825912:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.826119:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163b08 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x4874e45a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.826170:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35935:x1875453360476160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 734us (1365us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572545.826177:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4464 00010000:00010000:1.0:1788572545.826825:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.826835:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.826847:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.826852:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572545.826856:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.826864:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.826876:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:1.0:1788572545.826879:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572545.826882:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.826889:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ad7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.826911:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360476416 00000100:00100000:1.0:1788572545.826977:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360476416 00000100:00100000:1.0:1788572545.827072:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 944 00000100:00100000:1.0:1788572545.827083:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35936:x1875453360476416:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.827808:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35936:x1875453360476416:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 726us (2923us total) trans 4294970642 rc 0/0 00000100:00100000:1.0:1788572545.827816:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 944 00000100:00100000:3.0:1788572545.840386:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.840796:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.842360:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360476544 00000100:00100000:2.0:1788572545.842374:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360476544 00000100:00100000:2.0:1788572545.842452:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4465 00000100:00100000:2.0:1788572545.842472:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35935:x1875453360476544:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.842495:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.842519:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b0f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01da expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.842561:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.842577:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.842590:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.842709:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b0f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.842734:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 94 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.842744:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.842747:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572545.842751:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e200, rc 0) 00000100:00100000:2.0:1788572545.842818:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35935:x1875453360476544:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 349us (2443us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.842827:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4465 00000100:00100000:1.0:1788572545.845344:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360476672 00000100:00100000:1.0:1788572545.845359:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360476672 00000100:00100000:1.0:1788572545.845434:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1232 00000100:00100000:1.0:1788572545.845465:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360476672:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.845761:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360476672:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 317us (4965us total) trans 4294967629 rc 0/0 00000100:00100000:1.0:1788572545.845770:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1232 00000100:00100000:0.0:1788572545.861402:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.861578:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360476928 00000100:00100000:1.0:1788572545.861589:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360476928 00000100:00100000:1.0:1788572545.861680:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4466 00000100:00100000:1.0:1788572545.861695:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35936:x1875453360476928:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.861825:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35936:x1875453360476928:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 134us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.861833:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4466 00000100:00100000:1.0:1788572545.862346:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.862412:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360477184 00000100:00100000:0.0:1788572545.862418:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360477184 00000100:00100000:0.0:1788572545.862947:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 945 00000100:00100000:0.0:1788572545.862959:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35937:x1875453360477184:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.863359:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35937:x1875453360477184:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 401us (1013us total) trans 4294970643 rc 0/0 00000100:00100000:0.0:1788572545.863367:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 945 00000100:00100000:0.0:1788572545.874361:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.876723:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360477440 00000100:00100000:3.0:1788572545.876742:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360477440 00000100:00100000:3.0:1788572545.876822:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4467 00000100:00100000:3.0:1788572545.876833:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35935:x1875453360477440:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572545.879604:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.879632:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572545.879646:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572545.880862:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].69, 88 off13856 00000004:00010000:0.0:1788572545.881058:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b029c1e5 reply state 000000005fa701f9 transno 4294970644 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572545.881076:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572545.881250:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35935:x1875453360477440:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 4413us (6899us total) trans 4294970644 rc 0/0 00000100:00100000:0.0:1788572545.881260:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4467 00000100:00100000:3.0:1788572545.890971:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.891227:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360477568 00000100:00100000:2.0:1788572545.891239:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360477568 00000100:00100000:2.0:1788572545.891305:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4468 00000100:00100000:2.0:1788572545.891316:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35936:x1875453360477568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.891339:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.891364:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b24 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01e1 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.891456:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163b2b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.891480:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163b2b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.891492:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163b2b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.891532:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.891551:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163b2b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.891579:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b24 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e1 expref: 99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.891598:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163b2b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e1 expref: 98 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.891607:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572545.891610:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572545.891614:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:2.0:1788572545.891676:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35936:x1875453360477568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 362us (717us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.891687:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4468 00000100:00100000:0.0:1788572545.892102:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.892805:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360477824 00000100:00100000:3.0:1788572545.892815:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360477824 00000100:00100000:3.0:1788572545.892858:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4469 00000100:00100000:3.0:1788572545.892867:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35937:x1875453360477824:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.892971:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35937:x1875453360477824:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 103us (869us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.892978:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4469 00000100:00100000:0.0:1788572545.895382:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572545.897807:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.897827:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.897840:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.897846:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572545.897851:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.897859:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.897879:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:3.0:1788572545.897883:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1788572545.897889:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.897895:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163b1d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572545.898623:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360477952 00000100:00100000:2.0:1788572545.898635:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360477952 00000100:00100000:2.0:1788572545.898746:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4470 00000100:00100000:2.0:1788572545.898757:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35935:x1875453360477952:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.898785:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.898805:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01e8 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.898867:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 98 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.898881:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 98 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.898900:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.898903:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572545.898906:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:2.0:1788572545.899023:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35935:x1875453360477952:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 247us (3631us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.899031:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4470 00000100:00100000:1.0:1788572545.899298:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.900638:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.900659:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360478208 00000100:00100000:1.0:1788572545.900666:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360478208 00000100:00100000:1.0:1788572545.900720:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1233 00000100:00100000:1.0:1788572545.900732:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360478208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.900746:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.900764:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01f6 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572545.900790:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x53:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7381, blocks: 0x1 00010000:00010000:1.0:1788572545.900806:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=893s v=(0 0) 00010000:00010000:1.0:1788572545.900811:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.900820:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.900837:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 13 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.900849:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572545.900889:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360478208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 160us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.900896:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1233 00000100:00100000:2.0:1788572545.904856:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360478080 00000100:00100000:2.0:1788572545.904874:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360478080 00000100:00100000:2.0:1788572545.904995:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4471 00000100:00100000:2.0:1788572545.905024:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35937:x1875453360478080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.905048:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.905076:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b40 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01ef expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.905150:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163b47 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.905166:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163b47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.905179:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163b47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.905213:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.905233:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163b47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.905263:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b40 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01ef expref: 100 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.905284:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163b47 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01ef expref: 99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.905292:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:2.0:1788572545.905295:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572545.905299:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f200, rc 0) 00000100:00100000:2.0:1788572545.905373:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35937:x1875453360478080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 368us (6076us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.905381:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4471 00000100:00100000:2.0:1788572545.905788:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.906090:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360478336 00000100:00100000:1.0:1788572545.906096:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360478336 00000100:00100000:1.0:1788572545.906148:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1234 00000100:00100000:1.0:1788572545.906158:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360478336:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.906455:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360478336:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 286us (655us total) trans 4294967630 rc 0/0 00000100:00100000:1.0:1788572545.906519:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1234 00000100:00100000:3.0:1788572545.917999:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.921794:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360478464 00000100:00100000:1.0:1788572545.921808:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360478464 00000100:00100000:1.0:1788572545.921893:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1235 00000100:00100000:1.0:1788572545.921905:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360478464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.921930:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.921951:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f01fd expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572545.921979:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x52:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7381, blocks: 0x1 00010000:00010000:1.0:1788572545.921998:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=893s v=(0 0) 00010000:00010000:1.0:1788572545.922036:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.922046:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 13 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.922065:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 13 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.922079:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97400, rc 0) 00000100:00100000:1.0:1788572545.922133:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360478464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 232us (4148us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.922141:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1235 00000100:00100000:2.0:1788572545.922357:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.922488:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.922668:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360478720 00000100:00100000:3.0:1788572545.922674:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360478720 00000100:00100000:3.0:1788572545.922720:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4472 00000100:00100000:3.0:1788572545.922728:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35936:x1875453360478720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572545.922809:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35936:x1875453360478720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 81us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572545.922817:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4472 00000100:00100000:1.0:1788572545.923118:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360478976 00000100:00100000:1.0:1788572545.923124:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360478976 00000100:00100000:1.0:1788572545.923209:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 946 00000100:00100000:1.0:1788572545.923219:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35935:x1875453360478976:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.923609:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35935:x1875453360478976:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 391us (1118us total) trans 4294970645 rc 0/0 00000100:00100000:1.0:1788572545.923616:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 946 00000100:00100000:3.0:1788572545.928557:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572545.928659:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360479232 00000100:00100000:2.0:1788572545.928670:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360479232 00000100:00100000:2.0:1788572545.928738:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4473 00000100:00100000:2.0:1788572545.928750:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35937:x1875453360479232:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572545.928870:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35937:x1875453360479232:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 121us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.928879:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4473 00000100:00100000:3.0:1788572545.931498:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.931585:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360479488 00000100:00100000:1.0:1788572545.931594:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360479488 00000100:00100000:1.0:1788572545.931670:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4474 00000100:00100000:1.0:1788572545.931683:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35935:x1875453360479488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.931792:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35935:x1875453360479488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 113us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.931801:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4474 00000100:00100000:0.0:1788572545.943406:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572545.943460:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.943543:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360479616 00000100:00100000:1.0:1788572545.943554:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360479616 00000100:00100000:1.0:1788572545.943628:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1236 00000100:00100000:1.0:1788572545.943639:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35936:x1875453360479616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.943658:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.943676:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0204 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.943709:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.943719:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788572545.943724:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.943730:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 14 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.943766:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.943780:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.943796:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382375680:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.943903:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.944319:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.944642:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360479872 00000100:00100000:1.0:1788572545.944649:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360479872 00000100:00100000:1.0:1788572545.944707:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1237 00000100:00100000:1.0:1788572545.944716:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35937:x1875453360479872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.944729:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.944740:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 2/0,0 mode: --/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0212 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.944762:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.944769:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788572545.944772:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.944777:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 4/0,0 mode: --/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.944802:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.944811:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.944822:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382375808:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572545.944876:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9fa540 going to sleep for 11 seconds 00000100:00100000:3.0:1788572545.945513:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360479744 00000100:00100000:3.0:1788572545.945520:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360479744 00000100:00100000:1.0:1788572545.947923:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382375680:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572545.947973:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 14 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.948039:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572545.948098:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35936:x1875453360479616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4460us (4704us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.948107:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1236 00000100:00100000:2.0:1788572545.950214:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4475 00000100:00100000:2.0:1788572545.950229:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35935:x1875453360479744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572545.950254:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572545.950277:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163b63 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f020b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.950354:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b6a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.950369:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b6a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.950383:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b6a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572545.950419:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572545.950438:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b6a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.950478:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163b63 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f020b expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.950504:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b6a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f020b expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572545.950513:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572545.950516:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572545.950519:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:2.0:1788572545.950589:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35935:x1875453360479744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 363us (7127us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572545.950596:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4475 00000100:00100000:0.0:1788572545.956718:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.956826:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382375808:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572545.956876:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 3/0,0 mode: --/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.956899:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572545.956969:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35937:x1875453360479872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 12254us (12652us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.956979:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1237 00000100:00100000:3.0:1788572545.957662:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480128 00000100:00100000:3.0:1788572545.957673:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480128 00010000:00010000:3.0:1788572545.957720:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.957760:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1056 00000100:00100000:3.0:1788572545.957770:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360480128:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572545.957786:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572545.957815:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.957828:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.957853:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb163b4e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01fd expref: 14 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.957869:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.957874:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572545.957882:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.957892:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572545.957946:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572545.957961:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382375936:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572545.958066:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff57c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.958790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.958865:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480000 00000100:00100000:1.0:1788572545.958871:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480000 00010000:00010000:1.0:1788572545.958910:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.958955:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1057 00000100:00100000:1.0:1788572545.958965:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360480000:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572545.958977:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572545.958993:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.959038:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.959057:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb163b39 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01f6 expref: 14 pid: 13272 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.959071:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.959076:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.959083:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.959093:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.959123:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572545.959136:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382376064:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572545.959189:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9fa540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572545.969772:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.969886:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480256 00000100:00100000:1.0:1788572545.969896:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480256 00000100:00100000:1.0:1788572545.969980:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1238 00000100:00100000:1.0:1788572545.969990:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360480256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572545.970035:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572545.970061:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0219 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572545.970094:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x53:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7381, blocks: 0x1 00010000:00010000:1.0:1788572545.970116:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=894s v=(0 0) 00010000:00010000:1.0:1788572545.970121:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572545.970131:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 15 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.970148:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 15 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572545.970162:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572545.970320:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360480256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 329us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.970328:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1238 00000100:00100000:1.0:1788572545.972846:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382375936:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572545.972878:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572545.972946:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360480128:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 15177us (16241us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572545.972957:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1056 00000100:00100000:0.0:1788572545.973530:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382376064:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572545.973559:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572545.973598:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360480000:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 14635us (14811us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.973606:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1057 00000100:00100000:3.0:1788572545.975286:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.975369:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480384 00000100:00100000:1.0:1788572545.975377:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480384 00002000:00100000:1.0:1788572545.975433:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x52:0x0] (0->2) req@ffff9626c13bc380 x1875453360480384/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.975454:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bc380 with x1875453360480384 ext(0->2) 00002000:00010000:1.0:1788572545.975460:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bc380 00000100:00100000:1.0:1788572545.975492:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 279 00000100:00100000:1.0:1788572545.975504:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360480384:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572545.975973:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360480384:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 471us (693us total) trans 4294967631 rc 0/0 00000100:00100000:1.0:1788572545.975982:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 279 00002000:00100000:1.0:1788572545.975990:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x52:0x0] (0->2) req@ffff9626c13bc380 x1875453360480384/t4294967631(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/448 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572545.975997:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bc380 with x1875453360480384 ext(0->2) 00002000:00010000:1.0:1788572545.976000:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bc380 00000100:00100000:1.0:1788572545.979428:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572545.979492:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572545.979626:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480512 00000100:00100000:3.0:1788572545.979636:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480512 00000100:00100000:3.0:1788572545.979691:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480768 00000100:00100000:3.0:1788572545.979695:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480768 00000100:00100000:3.0:1788572545.979725:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4476 00000100:00100000:3.0:1788572545.979736:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35937:x1875453360480512:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.979894:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4477 00000100:00100000:0.0:1788572545.979905:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35935:x1875453360480768:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572545.979999:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35935:x1875453360480768:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 96us (506us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572545.980036:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4477 00000100:00100000:0.0:1788572545.993171:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.002867:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360480896 00000100:00100000:1.0:1788572546.002883:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360480896 00002000:00100000:1.0:1788572546.002956:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x53:0x0] (0->2) req@ffff9627e7874a80 x1875453360480896/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/0 e 0 to 0 dl 1788572556 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.002969:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7874a80 with x1875453360480896 ext(0->2) 00002000:00010000:1.0:1788572546.002975:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e7874a80 00000100:00100000:1.0:1788572546.003024:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 280 00000100:00100000:1.0:1788572546.003048:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360480896:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00080000:0.0:1788572546.003503:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572546.008907:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360480896:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 5863us (15736us total) trans 4294967633 rc 0/0 00000100:00100000:1.0:1788572546.008924:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 280 00002000:00100000:1.0:1788572546.008940:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x53:0x0] (0->2) req@ffff9627e7874a80 x1875453360480896/t4294967633(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:206/0 lens 496/448 e 0 to 0 dl 1788572556 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.008952:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7874a80 with x1875453360480896 ext(0->2) 00002000:00010000:1.0:1788572546.008960:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e7874a80 00000100:00100000:2.0:1788572546.009194:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.009228:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481024 00000100:00100000:2.0:1788572546.009237:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481024 00000100:00080000:2.0:1788572546.009263:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788572546.009309:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1239 00000100:00100000:2.0:1788572546.009318:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35935:x1875453360481024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.009334:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.009351:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0220 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.009382:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.009390:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.009395:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.009401:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.009436:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.009464:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.009483:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382376192:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572546.009568:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3180 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.033111:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382376192:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.033148:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 17 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.033171:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc 0) 00000100:00100000:1.0:1788572546.033229:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35935:x1875453360481024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 23911us (24034us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.033238:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1239 00000100:00100000:2.0:1788572546.036619:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.036660:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.036742:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481280 00000100:00100000:1.0:1788572546.036753:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481280 00000100:00100000:0.0:1788572546.036822:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481152 00000100:00100000:0.0:1788572546.036827:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481152 00010000:00010000:0.0:1788572546.036858:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.036864:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4478 00000100:00100000:0.0:1788572546.036886:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1058 00000100:00100000:0.0:1788572546.036894:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360481152:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572546.036910:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572546.036934:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.036949:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.036973:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f46fa957/0x85ace67feb163b71 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0219 expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.036990:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.036995:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.037048:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.037061:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.037100:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 16 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.037116:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382376320:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572546.037187:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f780 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.049000:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35936:x1875453360481280:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.049366:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.049467:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382376320:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572546.049492:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572546.049559:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360481152:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 12665us (12950us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.049571:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1058 00000100:00100000:1.0:1788572546.050996:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481408 00000100:00100000:1.0:1788572546.051047:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481408 00002000:00100000:1.0:1788572546.051113:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x53:0x0] (0->2) req@ffff9626c13bdf80 x1875453360481408/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.051125:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bdf80 with x1875453360481408 ext(0->2) 00002000:00010000:1.0:1788572546.051131:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bdf80 00000100:00100000:1.0:1788572546.051163:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 281 00000100:00100000:1.0:1788572546.051174:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360481408:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.051826:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360481408:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 651us (2458us total) trans 4294967632 rc 0/0 00000100:00100000:1.0:1788572546.051837:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 281 00002000:00100000:1.0:1788572546.051845:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x53:0x0] (0->2) req@ffff9626c13bdf80 x1875453360481408/t4294967632(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.051853:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bdf80 with x1875453360481408 ext(0->2) 00002000:00010000:1.0:1788572546.051857:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bdf80 00000100:00100000:2.0:1788572546.053041:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35936:x1875453360481280:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 16133us (16344us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.053054:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4478 00000100:00100000:1.0:1788572546.059811:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.059984:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481536 00000100:00100000:0.0:1788572546.059997:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481536 00000100:00100000:0.0:1788572546.060094:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4479 00000100:00100000:0.0:1788572546.060104:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35935:x1875453360481536:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.060228:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35935:x1875453360481536:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 125us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.060236:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4479 00000100:00100000:3.0:1788572546.064807:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.064846:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.068620:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481664 00000100:00100000:1.0:1788572546.068635:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481664 00000100:00100000:1.0:1788572546.068724:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481792 00000100:00100000:1.0:1788572546.068727:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481792 00000100:00100000:1.0:1788572546.068767:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1240 00000100:00100000:1.0:1788572546.068780:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360481664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.068849:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1241 00000100:00100000:1.0:1788572546.068857:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360481792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1788572546.092140:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970644 is committed 00000001:00080000:2.0:1788572546.092165:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970624 is committed 00000001:00080000:2.0:1788572546.092182:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970645 is committed 00000001:00080000:2.0:1788572546.092194:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970632 is committed 00000001:00080000:2.0:1788572546.092215:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970634 is committed 00000020:00080000:2.0:1788572546.092228:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ff5483c0 refcount 2 committed 0 result 0 batchid 128 00000020:00080000:2.0:1788572546.092233:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627fb363f00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff5486c0 00000020:00080000:2.0:1788572546.092237:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627fb363700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff028000 00000020:00080000:0.0:1788572546.092281:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 123 00000020:00080000:2.0:1788572546.092321:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 126 batchid updated 00000020:00080000:0.0:1788572546.092353:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 126 00000020:00080000:0.0:1788572546.092366:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 126 00000020:00080000:2.0:1788572546.092405:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c9b79a80 refcount 2 committed 0 result 0 batchid 129 00000020:00080000:2.0:1788572546.092408:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc100 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c9b799c0 00000020:00080000:2.0:1788572546.092411:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fccc00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffeb5c00 00000100:00100000:3.0:1788572546.092414:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35937:x1875453360480512:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 112675us (112990us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.092426:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4476 00000020:00080000:0.0:1788572546.092453:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 126 00000020:00080000:2.0:1788572546.092497:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627dffb33c0 refcount 2 committed 0 result 0 batchid 127 00000020:00080000:2.0:1788572546.092501:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc700 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627dffb3a80 00000020:00080000:2.0:1788572546.092505:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627caa5e000 00000020:00080000:0.0:1788572546.092548:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 129 committed batchid 126 00000020:00080000:0.0:1788572546.092808:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 126 00000100:00100000:3.0:1788572546.099582:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.099774:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360481920 00000100:00100000:0.0:1788572546.099784:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360481920 00000100:00100000:0.0:1788572546.099884:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1242 00000100:00100000:0.0:1788572546.099896:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360481920:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.100043:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360481920:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 148us (468us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.100051:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1242 00000100:00100000:1.0:1788572546.106800:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.106838:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360482176 00000100:00100000:1.0:1788572546.106850:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360482176 00000100:00100000:1.0:1788572546.107053:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4480 00000100:00100000:1.0:1788572546.107066:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35937:x1875453360482176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.107127:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.107156:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163b47 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01ef expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.107175:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:1.0:1788572546.107181:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572546.107211:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.107225:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.107235:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572546.107395:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000026c0f1ef reply state 00000000b10d26c4 transno 4294970646 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.107405:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.109591:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35937:x1875453360482176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2518us (2791us total) trans 4294970646 rc 0/0 00000100:00100000:1.0:1788572546.109603:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4480 00010000:00010000:0.0:1788572546.122132:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.122151:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.122160:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.122166:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.122172:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.122181:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.122209:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572546.122212:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.122218:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.122225:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163b7f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.122337:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.122401:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360482304 00000100:00100000:0.0:1788572546.122411:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360482304 00000100:00100000:0.0:1788572546.122518:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1243 00000100:00100000:0.0:1788572546.122530:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360482304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572546.122771:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x52:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572546.122775:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x52:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:0.0:1788572546.122778:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x52:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:0.0:1788572546.122780:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x52:0x0] updating lvb ctime from disk: 1788572545 -> 1788572524 00002000:00010000:0.0:1788572546.122782:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x52:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572546.122837:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+20:2350:x1875453360482304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 309us (502us total) trans 4294967633 rc 0/0 00000100:00100000:0.0:1788572546.122846:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1243 00000001:00080000:3.0:1788572546.128686:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967631 is committed 00000001:00080000:3.0:1788572546.128703:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967633 is committed 00000100:00100000:1.0:1788572546.128953:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360481792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 60090us (64099us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.128965:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1241 00000100:00100000:3.0:1788572546.134329:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.134472:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360482560 00000100:00100000:0.0:1788572546.134482:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360482560 00000100:00100000:0.0:1788572546.134543:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4481 00000100:00100000:0.0:1788572546.134553:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35936:x1875453360482560:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.134616:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572546.134647:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163b2b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e1 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.134666:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.134672:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572546.134704:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.134718:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.134728:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572546.134876:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000063186c44 reply state 00000000f325b5ca transno 4294970647 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.134886:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572546.134997:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35936:x1875453360482560:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 445us (676us total) trans 4294970647 rc 0/0 00000100:00100000:0.0:1788572546.135042:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4481 00000100:00100000:2.0:1788572546.137649:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360481664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 68863us (72842us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.137663:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1240 00000001:00080000:0.0:1788572546.137731:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967628 is committed 00000001:00080000:0.0:1788572546.137746:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967632 is committed 00000100:00100000:3.0:1788572546.153657:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.153811:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360482816 00000100:00100000:2.0:1788572546.153823:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360482816 00000100:00100000:2.0:1788572546.153958:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4482 00000100:00100000:2.0:1788572546.153971:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35937:x1875453360482816:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.154094:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.154114:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.154126:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 4/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.154162:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.154178:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 66 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.154645:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572546.154698:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.154708:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.154716:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.154722:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.154727:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.154735:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.154752:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.154756:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.154761:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.154768:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163b86 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.154838:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382376448:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572546.154947:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.158250:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.159688:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483200 00000100:00100000:0.0:1788572546.159702:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483200 00000100:00100000:0.0:1788572546.159765:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1244 00000100:00100000:0.0:1788572546.159776:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360483200:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572546.160081:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x53:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572546.160086:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x53:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:0.0:1788572546.160089:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x53:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:0.0:1788572546.160092:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x53:0x0] updating lvb ctime from disk: 1788572545 -> 1788572523 00002000:00010000:0.0:1788572546.160094:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x53:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572546.160156:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360483200:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 381us (1918us total) trans 4294967634 rc 0/0 00000100:00100000:0.0:1788572546.160164:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1244 00000100:00100000:1.0:1788572546.163100:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483072 00000100:00100000:1.0:1788572546.163115:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483072 00000100:00100000:0.0:1788572546.163329:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.163386:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4483 00000100:00100000:0.0:1788572546.163396:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35935:x1875453360483072:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.163468:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572546.163494:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163b6a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f020b expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.163516:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:0.0:1788572546.163522:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572546.163559:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.163574:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.163586:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572546.163757:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000099786fa3 reply state 000000005c9f8343 transno 4294970648 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.163767:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572546.163856:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35935:x1875453360483072:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 461us (9212us total) trans 4294970648 rc 0/0 00000100:00100000:0.0:1788572546.163863:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4483 00000100:00100000:3.0:1788572546.164490:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382376448:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572546.164521:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.164711:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483328 00000100:00100000:1.0:1788572546.164718:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483328 00010000:00010000:1.0:1788572546.164762:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 67 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.164798:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1059 00000100:00100000:1.0:1788572546.164809:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360483328:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.164821:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.164827:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 67 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.164839:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 67 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.164854:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c2477200/0x85ace67feb163abb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x96:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01be expref: 67 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.164868:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:1.0:1788572546.164872:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.164880:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.164887:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.164908:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.164957:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360483328:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 151us (1633us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.164964:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1059 00000100:00100000:0.0:1788572546.165085:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572546.165308:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165316:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165341:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165349:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165355:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165361:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.165367:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165375:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165385:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:3.0:1788572546.165388:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572546.165392:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165398:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163b8d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.165460:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35937:x1875453360482816:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 11493us (11818us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572546.165470:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4482 00000100:00100000:3.0:1788572546.165489:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483584 00000100:00100000:3.0:1788572546.165494:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483584 00000100:00100000:3.0:1788572546.165523:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4484 00000100:00100000:3.0:1788572546.165530:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35936:x1875453360483584:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.165585:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165596:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165604:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165633:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.165646:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 65 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.165661:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c13bc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382376576:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.165716:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f643c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572546.166906:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.166931:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.166939:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.166949:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.166953:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.166961:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.166976:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:1.0:1788572546.166979:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.166983:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.166989:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163b94 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.168739:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.168812:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483712 00000100:00100000:0.0:1788572546.168822:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483712 00000100:00100000:0.0:1788572546.168881:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1245 00000100:00100000:0.0:1788572546.168892:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360483712:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572546.169166:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x53:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572546.169170:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x53:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:0.0:1788572546.169173:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x53:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:0.0:1788572546.169176:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x53:0x0] updating lvb ctime from disk: 1788572545 -> 1788572523 00002000:00010000:0.0:1788572546.169179:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x53:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572546.169247:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360483712:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 356us (516us total) trans 4294967634 rc 0/0 00000100:00100000:0.0:1788572546.169256:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1245 00010000:00010000:3.0:1788572546.173250:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572546.175594:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.175657:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.175695:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.175724:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360484224 00000100:00100000:0.0:1788572546.175734:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360484224 00000100:00100000:0.0:1788572546.175788:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4485 00000100:00100000:0.0:1788572546.175799:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35935:x1875453360484224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.175871:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.175889:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.175900:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.175937:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.175951:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 65 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.175966:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382376704:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572546.176072:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bda180 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.176227:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483968 00000100:00100000:2.0:1788572546.176232:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483968 00010000:00010000:2.0:1788572546.176256:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 66 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.176286:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1060 00000100:00100000:2.0:1788572546.176294:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360483968:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.176304:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.176311:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 66 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.176324:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 66 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.176343:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572546.176348:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.176358:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.176365:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.176380:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.176414:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360483968:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 120us (756us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.176422:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1060 00010000:00010000:3.0:1788572546.176564:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb163a91 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x97:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b0 expref: 65 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.176582:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c13bc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382376576:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572546.176600:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176610:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176617:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176645:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176652:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176660:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176665:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.176670:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176678:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176688:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:3.0:1788572546.176690:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572546.176694:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.176700:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163b9b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.176754:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35936:x1875453360483584:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 11224us (11669us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572546.176763:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4484 00000100:00100000:1.0:1788572546.176954:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360483840 00000100:00100000:1.0:1788572546.176959:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360483840 00000100:00100000:1.0:1788572546.177034:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1246 00000100:00100000:1.0:1788572546.177045:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360483840:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.177077:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.177094:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.177102:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.177107:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.177111:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.177136:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 13 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.177146:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 13 pid: 6681 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.177158:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382376832:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572546.177195:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.183324:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.183539:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360484480 00000100:00100000:3.0:1788572546.183549:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360484480 00000100:00100000:3.0:1788572546.183608:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1247 00000100:00100000:3.0:1788572546.183618:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360484480:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.183662:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.183684:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.183693:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.183697:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.183702:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.183740:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.183754:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 11 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.183775:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382376960:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.183840:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f643c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.184428:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382376832:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.184583:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.184723:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382376704:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572546.184742:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.188160:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.188215:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.188342:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360484352 00000100:00100000:2.0:1788572546.188352:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360484352 00010000:00010000:2.0:1788572546.188392:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 64 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.188429:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360484608 00000100:00100000:2.0:1788572546.188432:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360484608 00010000:00010000:2.0:1788572546.188466:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 14 pid: 6681 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.188499:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1061 00000100:00100000:2.0:1788572546.188509:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360484352:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.188524:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.188532:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 64 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.188544:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 64 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.188564:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163b16 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x98:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01da expref: 64 pid: 5837 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.188580:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572546.188586:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.188597:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.188605:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.188677:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1062 00000100:00100000:3.0:1788572546.188688:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360484608:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.188698:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.188722:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 14 pid: 6681 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188734:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 14 pid: 6681 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188750:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0b7d0ec/0x85ace67feb163b5c lrc: 0/0,0 mode: --/PW res: [0x280000400:0x52:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0212 expref: 14 pid: 6681 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188763:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.188768:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.188773:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188784:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188806:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.188858:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360484608:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 172us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.188866:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1062 00010000:00010000:3.0:1788572546.188895:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188905:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188940:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188950:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188958:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188966:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572546.188971:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188981:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.188994:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.189050:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb163ba9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.189107:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360483840:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 12065us (13524us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.189116:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1246 00010000:00010000:0.0:1788572546.189553:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189562:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189590:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189596:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189602:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189607:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.189611:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189618:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189628:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:0.0:1788572546.189631:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572546.189636:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.189642:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163ba2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.189693:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35935:x1875453360484224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13895us (13996us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572546.189703:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4485 00010000:00010000:2.0:1788572546.189794:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.189829:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2349:x1875453360484352:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1320us (1677us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.189837:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1061 00000100:00100000:0.0:1788572546.192174:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382376960:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572546.192202:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.195255:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.196169:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360484736 00000100:00100000:2.0:1788572546.196181:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360484736 00010000:00010000:3.0:1788572546.196823:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 12 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.196922:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1063 00000100:00100000:3.0:1788572546.196933:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360484736:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.196949:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.196972:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 12 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.196987:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 12 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.197175:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163b55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0204 expref: 12 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.197201:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.197206:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.197212:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.197223:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.197299:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.197356:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360484736:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 424us (2110us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.197365:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1063 00010000:00010000:1.0:1788572546.198564:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198579:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198613:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198622:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198637:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198645:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572546.198650:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198659:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198676:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.198684:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163bb0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.198845:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360484480:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 15226us (15529us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.198855:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1247 00000100:00100000:0.0:1788572546.199208:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.199489:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360484992 00000100:00100000:3.0:1788572546.199498:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360484992 00000100:00100000:3.0:1788572546.199554:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4486 00000100:00100000:3.0:1788572546.199563:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35937:x1875453360484992:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.199641:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.199659:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.199669:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.200124:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: --/EX res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.200138:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 63 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.200146:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 4/0,1 mode: --/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.200183:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 63 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.200199:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 63 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.200214:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382377088:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.200284:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f64600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.202396:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.202968:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382377088:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.202996:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: --/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.203281:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360485120 00000100:00100000:3.0:1788572546.203288:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360485120 00000100:00100000:3.0:1788572546.203377:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1248 00000100:00100000:3.0:1788572546.203388:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360485120:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.203429:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.203511:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.203522:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.203527:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.203531:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.203566:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.203584:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 12 pid: 6682 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.203597:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382377216:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.203673:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.205261:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.205361:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360485248 00000100:00100000:1.0:1788572546.205369:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360485248 00010000:00010000:1.0:1788572546.205528:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 64 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.205613:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1064 00000100:00100000:1.0:1788572546.205627:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360485248:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.205641:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.205649:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 64 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.205662:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 64 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.205684:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163afa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01d3 expref: 64 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.205700:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:1.0:1788572546.205706:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: --/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.205716:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.205725:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.205762:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.205820:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2350:x1875453360485248:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 197us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.205828:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1064 00010000:00010000:3.0:1788572546.205883:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.205893:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572546.206150:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970649 00000001:00080000:3.0:1788572546.206329:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970650 00000001:00080000:3.0:1788572546.206383:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970651 00000100:00100000:3.0:1788572546.206497:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382377344:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572546.206559:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572546.206591:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.206728:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382377344 00000100:00100000:0.0:1788572546.206735:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382377344 00000100:00100000:0.0:1788572546.206798:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:0.0:1788572546.206807:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382377344:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572546.206883:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382377344:12345-0@lo:1000:mdt00_004.0 Request processed in 78us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.206891:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:0.0:1788572546.207055:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967628 gen 2 00000004:00080000:0.0:1788572546.207061:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967627 committed transno 4294967628 00000100:00100000:0.0:1788572546.207089:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382377344:0@lo:1000:mdt00_004.0 00000001:00080000:0.0:1788572546.207493:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970652 00000100:00100000:0.0:1788572546.207553:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9d180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382377472:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572546.207585:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572546.207605:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f64d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.207621:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382377472 00000100:00100000:0.0:1788572546.207625:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382377472 00000100:00100000:0.0:1788572546.207652:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:0.0:1788572546.207659:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382377472:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572546.207811:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382377472:12345-0@lo:1000:mdt00_004.0 Request processed in 152us (225us total) trans 4294967635 rc 0/0 00000100:00100000:0.0:1788572546.207820:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:0.0:1788572546.207853:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9d180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382377472:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1788572546.207881:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207892:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207900:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207907:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.207912:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207921:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207938:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9627fb1e7600) 00010000:00010000:0.0:1788572546.207941:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572546.207946:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 2/0,0 mode: --/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207953:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163bbe lrc: 0/0,0 mode: --/EX res: [0x200000402:0x96:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207983:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.207995:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208000:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208027:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.208030:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208036:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208045:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627c29da300) 00010000:00010000:0.0:1788572546.208047:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572546.208051:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208057:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93200/0x85ace67feb15fb36 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x75:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208081:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208092:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208101:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208108:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208115:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208120:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208125:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.208128:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208133:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208140:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627c29da300) 00010000:00010000:0.0:1788572546.208142:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.208144:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208149:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163bcc lrc: 0/0,0 mode: --/EX res: [0x200000401:0x75:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208158:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208163:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208169:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208174:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.208177:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208183:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208192:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627c29da300) 00010000:00010000:0.0:1788572546.208194:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572546.208197:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.208204:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163bb7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x75:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.208351:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35937:x1875453360484992:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 8787us (9152us total) trans 4294970653 rc 0/0 00000100:00100000:0.0:1788572546.208361:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4486 00000020:00080000:0.0:1788572546.208365:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00080000:3.0:1788572546.208758:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788572546.212434:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.212492:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.212596:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360485376 00000100:00100000:1.0:1788572546.212606:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360485376 00010000:00010000:1.0:1788572546.212662:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 13 pid: 6682 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.212713:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1065 00000100:00100000:1.0:1788572546.212726:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360485376:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.212741:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.212765:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 13 pid: 6682 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.212780:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 13 pid: 6682 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.212805:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.212810:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.212816:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.212826:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.212850:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.212901:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360485376:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 179us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.212909:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1065 00000100:00100000:2.0:1788572546.212988:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360485632 00000100:00100000:2.0:1788572546.212993:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360485632 00000100:00100000:2.0:1788572546.213053:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4487 00000100:00100000:2.0:1788572546.213061:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35936:x1875453360485632:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.213135:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x70:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213149:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213159:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213414:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: --/EX res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213426:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213437:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 4/0,1 mode: --/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213594:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.213610:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 67 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.213627:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382377600:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572546.213683:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7b40 going to sleep for 11 seconds 00010000:00010000:3.0:1788572546.214364:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb163b78 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x53:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0220 expref: 12 pid: 6682 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.214381:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382377216:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572546.214399:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214412:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214421:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214549:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214561:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214569:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214576:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572546.214581:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214590:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214605:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.214613:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef924bbe/0x85ace67feb163bc5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.214728:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360485120:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11340us (12339us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.214737:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1248 00000100:00100000:0.0:1788572546.221113:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.221215:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360485888 00000100:00100000:3.0:1788572546.221225:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360485888 00000100:00100000:3.0:1788572546.221275:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 947 00000100:00100000:3.0:1788572546.221284:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35937:x1875453360485888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572546.222191:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x96:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572546.222639:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970655 00010000:00010000:3.0:1788572546.222695:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.222714:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.224094:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.224106:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.224112:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.224118:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.224802:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572546.224947:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35937:x1875453360485888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 3659us (3840us total) trans 4294970656 rc 0/0 00000100:00100000:3.0:1788572546.224957:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 947 00000020:00080000:3.0:1788572546.224963:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788572546.225990:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572546.226029:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.226038:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572546.226043:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.226052:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.226067:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x96:0x0].0x0 (ffff9626c1d8f900) 00010000:00010000:1.0:1788572546.226070:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572546.226075:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.226081:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163be1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x96:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.226102:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572546.226104:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572546.226106:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572546.226448:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382377600:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572546.226487:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: --/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.227386:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.227722:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.227862:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360486016 00000100:00100000:2.0:1788572546.227871:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360486016 00010000:00010000:2.0:1788572546.227907:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 70 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.227956:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1066 00000100:00100000:2.0:1788572546.227966:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2348:x1875453360486016:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.227979:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.227986:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 70 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.228029:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 70 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.228050:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb163aad lrc: 0/0,0 mode: --/CR res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01b7 expref: 70 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.228068:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572546.228073:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: --/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.228082:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.228089:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.228120:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.228167:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2348:x1875453360486016:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 202us (446us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.228175:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1066 00010000:00010000:3.0:1788572546.228317:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.228326:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572546.228550:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970654 00000001:00080000:3.0:1788572546.228707:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970657 00000001:00080000:3.0:1788572546.228753:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970658 00000100:00100000:3.0:1788572546.228832:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382377728:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572546.228883:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572546.228907:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.229095:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382377728 00000100:00100000:0.0:1788572546.229100:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382377728 00000100:00100000:0.0:1788572546.229152:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1788572546.229161:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382377728:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572546.229242:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382377728:12345-0@lo:1000:mdt00_001.0 Request processed in 82us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.229249:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1788572546.229617:0:5838:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967631 gen 2 00000004:00080000:0.0:1788572546.229624:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967630 committed transno 4294967631 00000004:00080000:0.0:1788572546.229642:0:5838:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967631 committed transno 4294967631 00000100:00100000:0.0:1788572546.229656:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382377728:0@lo:1000:mdt00_001.0 00000001:00080000:0.0:1788572546.230145:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970659 00000100:00100000:0.0:1788572546.230324:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382377856:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572546.230363:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572546.230393:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077180 going to sleep for 11 seconds 00000100:00080000:3.0:1788572546.230512:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572546.230586:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360486272 00000100:00100000:1.0:1788572546.230593:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360486272 00000100:00100000:1.0:1788572546.230645:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4488 00000100:00100000:1.0:1788572546.230655:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35935:x1875453360486272:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.230728:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.230745:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.230757:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.231093:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: --/EX res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.231105:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 74 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.231113:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 4/0,1 mode: --/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.231139:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 74 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.231152:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 74 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.231164:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382377984:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572546.231288:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.231543:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382377856 00000100:00100000:2.0:1788572546.231549:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382377856 00000100:00100000:3.0:1788572546.231621:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:3.0:1788572546.231630:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5838:x1875453382377856:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572546.231838:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382377856:12345-0@lo:1000:mdt00_001.0 Request processed in 209us (1474us total) trans 4294967635 rc 0/0 00000100:00100000:3.0:1788572546.231856:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:0.0:1788572546.232039:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382377856:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1788572546.232070:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232078:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232084:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232089:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.232094:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232101:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232115:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.232118:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572546.232123:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 2/0,0 mode: --/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232133:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163bda lrc: 0/0,0 mode: --/EX res: [0x200000402:0x97:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232163:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232169:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232175:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232180:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.232182:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232189:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232197:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:0.0:1788572546.232199:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572546.232297:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232305:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb15fb13 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x70:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232324:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x70:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232334:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232342:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232349:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232355:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232360:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232365:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.232368:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232373:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232380:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:0.0:1788572546.232382:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.232385:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232391:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163bf6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x70:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232400:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232407:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232413:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232419:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.232422:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232428:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232435:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:0.0:1788572546.232438:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572546.232452:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.232544:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163bd3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x70:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.232868:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35936:x1875453360485632:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 19804us (20371us total) trans 4294970660 rc 0/0 00000100:00100000:0.0:1788572546.232877:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4487 00000020:00080000:0.0:1788572546.232881:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572546.236129:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.236174:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.236236:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360486528 00000100:00100000:3.0:1788572546.236236:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360486656 00000100:00100000:2.0:1788572546.236245:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360486528 00000100:00100000:3.0:1788572546.236245:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360486656 00010000:00010000:2.0:1788572546.236285:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 75 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.236303:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 948 00000100:00100000:3.0:1788572546.236313:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:35936:x1875453360486656:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.236316:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1067 00000100:00100000:2.0:1788572546.236322:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360486528:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.236336:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.236342:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 75 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.236355:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 75 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.236378:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:2.0:1788572546.236384:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: --/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.236393:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.236400:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.236418:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.236485:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360486528:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 162us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.236492:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1067 00010000:00010000:1.0:1788572546.236624:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163b32 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f01e8 expref: 75 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.236640:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382377984:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.236676:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.236685:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.236691:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572546.236695:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x97:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572546.236740:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970662 00010000:00010000:3.0:1788572546.236777:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572546.236781:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970661 00010000:00010000:3.0:1788572546.236788:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.236796:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.236803:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.236809:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.236816:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.236839:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572546.236899:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35936:x1875453360486656:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 587us (724us total) trans 4294970663 rc 0/0 00000100:00100000:3.0:1788572546.236907:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 948 00000001:00080000:1.0:1788572546.236917:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970664 00000020:00080000:3.0:1788572546.236922:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000001:00080000:1.0:1788572546.236962:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970665 00000100:00100000:1.0:1788572546.237082:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382378112:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572546.237126:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572546.237152:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.237178:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382378112 00000100:00100000:3.0:1788572546.237183:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382378112 00000100:00100000:3.0:1788572546.237221:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1788572546.237229:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382378112:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572546.237289:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382378112:12345-0@lo:1000:mdt00_000.0 Request processed in 60us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.237295:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1788572546.237324:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382378112:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1788572546.237497:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572546.237503:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.237510:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14ce00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572546.237514:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.237526:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.237537:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x97:0x0].0x0 (ffff9626c1d8fa00) 00010000:00010000:2.0:1788572546.237539:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572546.237543:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 1/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.237549:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163bfd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x97:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.237565:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572546.237567:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572546.237569:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1788572546.237797:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970666 00000100:00100000:3.0:1788572546.237894:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382378240:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572546.237929:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572546.237993:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.239812:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382378240 00000100:00100000:1.0:1788572546.239823:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382378240 00000100:00100000:0.0:1788572546.240100:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:0.0:1788572546.240112:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382378240:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572546.240405:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:5837:x1875453382378240:12345-0@lo:1000:mdt00_000.0 Request processed in 295us (2474us total) trans 4294967636 rc 0/0 00000100:00100000:0.0:1788572546.240417:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:1.0:1788572546.240809:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382378240:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1788572546.240859:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240869:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240876:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240883:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.240887:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240895:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240912:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9627e7a95f00) 00010000:00010000:1.0:1788572546.240916:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572546.240920:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 2/0,0 mode: --/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240931:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163bef lrc: 0/0,0 mode: --/EX res: [0x200000402:0x98:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240966:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240973:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240978:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240981:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.240983:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240987:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.240993:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627d55e6f00) 00010000:00010000:1.0:1788572546.240994:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572546.240996:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241035:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb15faf0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x72:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241059:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241073:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241083:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241088:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241092:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241095:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241098:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.241100:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241103:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241107:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627d55e6f00) 00010000:00010000:1.0:1788572546.241109:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.241110:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241113:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c04 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x72:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241119:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241122:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241126:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241129:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.241131:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241134:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241137:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627d55e6f00) 00010000:00010000:1.0:1788572546.241138:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572546.241140:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.241143:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163be8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x72:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.241398:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35935:x1875453360486272:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 10741us (14016us total) trans 4294970667 rc 0/0 00000100:00100000:0.0:1788572546.241408:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4488 00000020:00080000:0.0:1788572546.241412:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1788572546.242288:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572546.242295:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572546.242330:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 157 count 101 00000040:00080000:1.0:1788572546.242338:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:1.0:1788572546.242941:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382378368:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572546.243050:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.245451:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.245534:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382378368:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572546.245755:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360486912 00000100:00100000:3.0:1788572546.245763:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360486912 00000100:00100000:3.0:1788572546.245861:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 949 00000100:00100000:3.0:1788572546.245872:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35935:x1875453360486912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572546.246357:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x98:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572546.246414:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970668 00010000:00010000:3.0:1788572546.246487:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.246504:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.246517:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.246526:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.246531:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.246538:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.246553:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572546.246648:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35935:x1875453360486912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 775us (1211us total) trans 4294970669 rc 0/0 00000100:00100000:3.0:1788572546.246657:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 949 00000020:00080000:3.0:1788572546.246662:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572546.246730:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572546.246738:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.246746:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572546.246750:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.246759:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.246772:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x98:0x0].0x0 (ffff9626c1d8f300) 00010000:00010000:0.0:1788572546.246775:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572546.246780:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 1/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.246786:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163c0b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x98:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.246805:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572546.246807:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572546.246809:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572546.248864:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.248937:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487040 00000100:00100000:1.0:1788572546.248947:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487040 00000100:00100000:1.0:1788572546.249052:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4489 00000100:00100000:1.0:1788572546.249065:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35940:x1875453360487040:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572546.249099:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6e:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572546.249215:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35940:x1875453360487040:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 154us (358us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.249223:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4489 00000100:00100000:1.0:1788572546.255603:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.255956:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487168 00000100:00100000:0.0:1788572546.255966:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487168 00000100:00100000:0.0:1788572546.256367:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4490 00000100:00100000:0.0:1788572546.256379:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35939:x1875453360487168:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572546.256419:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x74:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572546.256547:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35939:x1875453360487168:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 170us (953us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.256555:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4490 00000100:00100000:2.0:1788572546.257146:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.257199:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.257308:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487296 00000100:00100000:1.0:1788572546.257314:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487296 00000100:00100000:1.0:1788572546.257369:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487424 00000100:00100000:1.0:1788572546.257372:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487424 00000100:00100000:1.0:1788572546.257391:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4491 00000100:00100000:1.0:1788572546.257399:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35938:x1875453360487296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572546.257420:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x71:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572546.257501:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35938:x1875453360487296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 102us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.257509:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4491 00000100:00100000:1.0:1788572546.257523:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4492 00000100:00100000:1.0:1788572546.257532:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35940:x1875453360487424:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.257589:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257607:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257619:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257634:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257641:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257648:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257670:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257679:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257685:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572546.257714:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6e:0x0]: returning size 3 00000004:00200000:1.0:1788572546.257719:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572546.257734:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257747:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257753:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257761:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257767:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.257771:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257780:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257792:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572546.257795:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.257799:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257806:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163c12 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257819:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257825:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257831:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257836:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.257839:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257844:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257863:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x15eaf752 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.257865:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.257868:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257877:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c19 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x15eaf752 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257889:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257895:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257900:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257905:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.257907:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257912:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257919:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c29dab00) 00010000:00010000:1.0:1788572546.257921:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.257924:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.257929:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c20 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.257968:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35940:x1875453360487424:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 438us (768us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.257975:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4492 00000100:00100000:2.0:1788572546.261887:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.262044:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487552 00000100:00100000:0.0:1788572546.262054:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487552 00000100:00100000:0.0:1788572546.262116:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4493 00000100:00100000:0.0:1788572546.262126:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35938:x1875453360487552:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.262205:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262222:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262233:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262248:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262256:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262262:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262288:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262297:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262303:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.262337:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x71:0x0]: returning size 3 00000004:00200000:0.0:1788572546.262340:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.262352:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262362:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262369:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262376:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262382:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.262385:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262393:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262404:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572546.262406:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.262410:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262421:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163c27 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262434:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262451:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262457:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262462:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.262465:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262470:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262477:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x6b77973c (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.262479:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.262481:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262487:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163c2e lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x6b77973c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262499:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262505:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262512:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262517:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.262520:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262525:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262531:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627d55e6200) 00010000:00010000:0.0:1788572546.262533:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.262536:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.262541:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163c35 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.262608:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35938:x1875453360487552:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 484us (731us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.262616:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4493 00000100:00100000:2.0:1788572546.266926:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.266971:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.267097:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487680 00000100:00100000:3.0:1788572546.267108:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487680 00000100:00100000:3.0:1788572546.267165:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360487936 00000100:00100000:3.0:1788572546.267169:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360487936 00000100:00100000:3.0:1788572546.267190:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4494 00000100:00100000:3.0:1788572546.267199:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35939:x1875453360487680:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.267276:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267294:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267307:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267323:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267330:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267337:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267363:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267372:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267383:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.267423:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x74:0x0]: returning size 3 00000004:00200000:3.0:1788572546.267427:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.267449:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267461:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267466:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267473:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267478:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.267482:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267490:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267499:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572546.267502:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.267506:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267512:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163c3c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267527:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267533:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267538:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267542:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.267544:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267549:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267554:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xf3c95776 (ffff9626c1d8fd00) 00010000:00010000:3.0:1788572546.267556:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.267559:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267563:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163c43 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xf3c95776 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267577:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267583:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267587:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.267592:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.267595:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.267657:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4495 00000100:00100000:0.0:1788572546.267668:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35938:x1875453360487936:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.267679:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.267692:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163c51 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0227 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267770:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267779:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267787:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.267796:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x71:0x0]: returning size 3 00010000:00010000:0.0:1788572546.267892:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267900:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267905:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267910:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.267912:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267919:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267927:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627d55e6200) 00010000:00010000:0.0:1788572546.267930:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572546.267933:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267939:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163c58 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267976:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163c51 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0227 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.267987:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572546.267989:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572546.267993:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163c51 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0227 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.268020:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc200, rc 0) 00000100:00100000:0.0:1788572546.268081:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35938:x1875453360487936:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 415us (1108us total) trans 4294970670 rc 301/301 00000100:00100000:0.0:1788572546.268089:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4495 00010000:00010000:3.0:1788572546.268569:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.268579:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc16d00) 00010000:00010000:3.0:1788572546.268581:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.268585:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.268590:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163c4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.268633:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35939:x1875453360487680:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1434us (1719us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.268640:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4494 00000100:00100000:2.0:1788572546.270903:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.271027:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360488320 00000100:00100000:0.0:1788572546.271037:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360488320 00000100:00100000:0.0:1788572546.271087:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4496 00000100:00100000:0.0:1788572546.271097:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35938:x1875453360488320:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572546.271129:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x71:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572546.271241:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35938:x1875453360488320:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 145us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.271250:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4496 00000100:00100000:1.0:1788572546.271633:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.271739:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360488192 00000100:00100000:3.0:1788572546.271745:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360488192 00000100:00100000:3.0:1788572546.271787:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4497 00000100:00100000:3.0:1788572546.271796:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35940:x1875453360488192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.271808:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.271828:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163c66 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f022e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.271919:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.271938:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.271949:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.271960:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6e:0x0]: returning size 3 00010000:00010000:3.0:1788572546.272085:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272095:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272102:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272108:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.272113:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272120:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272132:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c29dab00) 00010000:00010000:3.0:1788572546.272135:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572546.272139:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272145:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163c6d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272193:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163c66 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f022e expref: 83 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272204:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572546.272207:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572546.272211:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163c66 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f022e expref: 83 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.272221:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d800, rc 0) 00000100:00100000:3.0:1788572546.272255:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35940:x1875453360488192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 460us (624us total) trans 4294970671 rc 301/301 00000100:00100000:3.0:1788572546.272262:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4497 00000100:00100000:1.0:1788572546.273229:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.273487:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360488576 00000100:00100000:2.0:1788572546.273495:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360488576 00000100:00100000:2.0:1788572546.273548:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4498 00000100:00100000:2.0:1788572546.273558:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35939:x1875453360488576:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.273577:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.273595:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163c7b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0235 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273688:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273702:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273714:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572546.273725:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x74:0x0]: returning size 3 00010000:00010000:2.0:1788572546.273811:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273819:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273826:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273832:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.273837:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273844:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273857:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc16d00) 00010000:00010000:2.0:1788572546.273860:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572546.273864:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273875:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163c82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273914:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163c7b lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0235 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273924:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572546.273926:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572546.273930:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163c7b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0235 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.273940:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc 0) 00000100:00100000:2.0:1788572546.274000:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35939:x1875453360488576:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 443us (776us total) trans 4294970672 rc 301/301 00000100:00100000:2.0:1788572546.274037:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4498 00000100:00100000:0.0:1788572546.276816:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.277495:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360488704 00000100:00100000:1.0:1788572546.277506:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360488704 00000100:00100000:1.0:1788572546.277610:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4499 00000100:00100000:1.0:1788572546.277623:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35938:x1875453360488704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.277645:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.277674:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163c90 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f023c expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.277743:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.277759:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.277770:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.277784:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.277792:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.277799:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278175:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163ca5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278196:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163ca5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278203:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163ca5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572546.278233:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:1.0:1788572546.278237:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572546.278258:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163ca5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278271:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278277:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278285:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278291:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.278295:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278302:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278313:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572546.278316:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.278321:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278331:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163c97 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278341:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278347:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278352:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278356:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.278359:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278366:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278373:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.278376:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.278378:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278384:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163c9e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278412:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163c90 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f023c expref: 85 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278424:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163ca5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f023c expref: 84 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.278429:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572546.278431:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572546.278432:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96000, rc 0) 00000100:00100000:1.0:1788572546.278511:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35938:x1875453360488704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 891us (1702us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.278518:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4499 00000100:00100000:0.0:1788572546.280287:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.280345:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.280391:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360488832 00000100:00100000:1.0:1788572546.280401:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360488832 00000100:00100000:1.0:1788572546.280466:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360488960 00000100:00100000:1.0:1788572546.280469:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360488960 00000100:00100000:1.0:1788572546.280494:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4500 00000100:00100000:1.0:1788572546.280505:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35939:x1875453360488832:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572546.280538:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x74:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572546.280626:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4501 00000100:00100000:0.0:1788572546.280636:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35940:x1875453360488960:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.280638:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35939:x1875453360488832:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 137us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.280645:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4500 00000004:00000080:0.0:1788572546.280657:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x6e:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572546.280706:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35940:x1875453360488960:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 72us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.280713:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4501 00000100:00100000:1.0:1788572546.283259:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.283356:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.283542:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360489216 00000100:00100000:2.0:1788572546.283552:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360489216 00000100:00100000:1.0:1788572546.283568:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360489344 00000100:00100000:1.0:1788572546.283573:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360489344 00000100:00100000:2.0:1788572546.283612:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4502 00000100:00100000:2.0:1788572546.283621:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35938:x1875453360489216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.283659:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4503 00000100:00100000:1.0:1788572546.283670:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35940:x1875453360489344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.283679:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.283703:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163cac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f024a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.283762:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.283765:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572546.283783:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.283790:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163ca5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f023c expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.283797:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.283805:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572546.283808:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.283810:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572546.283817:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163cba lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0243 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.283821:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.283828:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.283833:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284162:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163cc8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284172:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284180:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572546.284271:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:1.0:1788572546.284275:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572546.284294:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284305:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284311:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284317:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284322:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.284326:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284334:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284342:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572546.284345:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.284348:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284354:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163cb3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284363:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284369:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284373:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284378:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.284381:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284386:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284392:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.284395:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.284397:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284402:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163cc1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284432:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163cac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f024a expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572546.284454:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x54:0x0], index 0: 0 00010000:00010000:1.0:1788572546.284457:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163cc8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f024a expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.284633:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ccf lrc: 3/1,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.284642:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ccf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.284650:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ccf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.284661:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572546.284664:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572546.284666:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97400, rc 0) 00000100:00100000:1.0:1788572546.284770:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35940:x1875453360489344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1103us (1413us total) trans 0 rc 0/0 00010000:00010000:2.0:1788572546.284770:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163cba lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0243 expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.284778:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4503 00010000:00010000:2.0:1788572546.284783:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ccf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0243 expref: 87 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.284790:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572546.284791:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572546.284794:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:2.0:1788572546.284815:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35938:x1875453360489216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1195us (1564us total) trans 4294970673 rc 0/0 00000100:00100000:2.0:1788572546.284821:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4502 00000100:00100000:1.0:1788572546.287850:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.288164:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360489600 00000100:00100000:2.0:1788572546.288175:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360489600 00000100:00100000:2.0:1788572546.288239:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4504 00000100:00100000:2.0:1788572546.288249:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35940:x1875453360489600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.288280:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.288285:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572546.288312:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163cc8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f024a expref: 86 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.288332:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572546.288337:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572546.288350:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163cdd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0251 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572546.288786:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x54:0x0], index 1: 0 00010000:00010000:2.0:1788572546.289158:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ce4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.289171:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ce4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.289179:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ce4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.289232:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163cdd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0251 expref: 88 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.289246:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ce4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0251 expref: 87 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.289254:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:2.0:1788572546.289256:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572546.289258:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:3.0:1788572546.289430:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.289486:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360489728 00000100:00100000:0.0:1788572546.289495:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360489728 00000100:00100000:0.0:1788572546.289539:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4505 00000100:00100000:0.0:1788572546.289549:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35939:x1875453360489728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.289561:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.289574:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163cf2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0258 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.289632:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.289643:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.289651:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.289656:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35940:x1875453360489600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1408us (1817us total) trans 4294970674 rc 0/0 00010000:00010000:0.0:1788572546.289663:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.289671:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.289672:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4504 00010000:00010000:0.0:1788572546.289677:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290063:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163d07 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290074:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163d07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290082:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163d07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.290102:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:0.0:1788572546.290105:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.290119:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163d07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290131:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290137:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290143:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290149:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.290153:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290160:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290170:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572546.290174:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.290178:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290184:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163cf9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290200:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290207:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290212:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290217:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.290219:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290225:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290232:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.290234:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.290237:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290243:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d00 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290270:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163cf2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0258 expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290280:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163d07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0258 expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.290285:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572546.290287:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572546.290288:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bca00, rc 0) 00000100:00100000:0.0:1788572546.290320:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35939:x1875453360489728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 772us (889us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.290326:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4505 00000100:00100000:3.0:1788572546.293432:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.293593:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.293887:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360490112 00000100:00100000:2.0:1788572546.293898:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360490112 00000100:00100000:2.0:1788572546.293971:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360489984 00000100:00100000:2.0:1788572546.293975:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360489984 00000100:00100000:2.0:1788572546.294044:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4506 00000100:00100000:2.0:1788572546.294054:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35938:x1875453360490112:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.294114:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572546.294141:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163ccf lrc: 0/0,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0243 expref: 89 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294159:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572546.294174:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb163d07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0258 expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294182:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572546.294185:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572546.294207:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294221:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294231:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294247:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294253:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.294259:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.294542:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4507 00000100:00100000:3.0:1788572546.294553:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35940:x1875453360489984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.294564:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.294580:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163d1c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f025f expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294640:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294650:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294673:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294686:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294693:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294699:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294896:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294907:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 2/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294914:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294930:0:9040:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.294940:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295070:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295079:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295087:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295093:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.295097:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295106:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295116:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572546.295119:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.295122:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295129:0:5839:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295137:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295153:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295161:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d0e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295171:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295178:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295185:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295190:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.295194:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295201:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295208:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x6963fd9f (ffff9627e7a95b00) 00010000:00010000:2.0:1788572546.295211:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.295214:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.295220:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d15 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x6963fd9f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.295333:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35938:x1875453360490112:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1279us (1906us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572546.295341:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4506 00010000:00010000:3.0:1788572546.295403:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295411:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.295437:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:3.0:1788572546.295452:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.295478:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295490:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295497:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295502:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295507:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.295510:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295515:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295524:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572546.295526:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.295529:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295535:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163d23 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295546:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295556:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295561:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295565:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.295568:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295573:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295579:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9626c1d8fd00) 00010000:00010000:3.0:1788572546.295582:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.295584:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295589:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d2a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295617:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163d1c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f025f expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295629:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f025f expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.295636:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572546.295638:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572546.295641:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14c400, rc 0) 00000100:00100000:3.0:1788572546.295666:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35940:x1875453360489984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1116us (2076us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.295673:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4507 00000100:00100000:1.0:1788572546.300313:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.301242:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360490496 00000100:00100000:2.0:1788572546.301254:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360490496 00000100:00100000:2.0:1788572546.301326:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4508 00000100:00100000:2.0:1788572546.301337:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35939:x1875453360490496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.301358:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.301381:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d38 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f026d expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572546.301965:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x55:0x0], index 0: 0 00010000:00010000:2.0:1788572546.302197:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d3f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.302213:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d3f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.302224:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d3f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.302283:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163d38 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f026d expref: 90 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.302303:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d3f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f026d expref: 89 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.302317:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572546.302320:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572546.302324:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc 0) 00000100:00100000:2.0:1788572546.302394:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35939:x1875453360490496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1059us (2091us total) trans 4294970675 rc 0/0 00000100:00100000:2.0:1788572546.302403:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4508 00000100:00100000:1.0:1788572546.303236:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.303382:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360490240 00000100:00100000:0.0:1788572546.303390:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360490240 00000100:00100000:0.0:1788572546.303455:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4509 00000100:00100000:0.0:1788572546.303594:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35938:x1875453360490240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.303612:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.303629:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163d4d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0266 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.303670:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.303682:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.303692:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.303786:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb163d4d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 91 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.303802:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.303811:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.303814:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572546.303817:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc600, rc 0) 00000100:00100000:0.0:1788572546.303867:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35938:x1875453360490240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 277us (634us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.304037:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4509 00000100:00100000:3.0:1788572546.312318:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.312407:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360490752 00000100:00100000:1.0:1788572546.312417:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360490752 00000100:00100000:1.0:1788572546.312502:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4510 00000100:00100000:1.0:1788572546.312514:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35940:x1875453360490752:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.312573:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572546.312600:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ce4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0251 expref: 90 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312618:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627e7a95700) 00010000:00010000:1.0:1788572546.312633:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163d31 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f025f expref: 89 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312642:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572546.312646:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572546.312681:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312696:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312707:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312721:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312728:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312735:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312972:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312981:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312988:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.312994:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.313000:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313048:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313059:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572546.313061:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.313065:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313071:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163d5b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313080:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313086:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313091:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313096:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.313098:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313104:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313112:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x5b56ef08 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.313114:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.313117:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.313123:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163d62 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x5b56ef08 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.313209:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35940:x1875453360490752:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 696us (901us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572546.313217:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4510 00000100:00100000:1.0:1788572546.313873:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.315272:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.316670:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.316689:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360490880 00000100:00100000:2.0:1788572546.316699:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360490880 00000100:00100000:2.0:1788572546.316786:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360491136 00000100:00100000:2.0:1788572546.316791:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360491136 00000100:00100000:2.0:1788572546.316820:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4511 00000100:00100000:2.0:1788572546.316832:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35939:x1875453360490880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.316871:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.316895:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163d69 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0274 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.316935:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4512 00000100:00100000:1.0:1788572546.316958:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35938:x1875453360491136:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.316967:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.316982:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.316993:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317025:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.317034:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317038:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.317043:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317044:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.317052:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317320:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163d85 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317329:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163d85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317335:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163d85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572546.317357:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:2.0:1788572546.317360:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572546.317377:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163d85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572546.317380:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].72, 88 off14048 00010000:00010000:2.0:1788572546.317388:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572546.317389:0:5839:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 894386384029ns, next at 895386381689ns, rc = 0 00010000:00010000:2.0:1788572546.317395:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317400:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317405:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.317409:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317417:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317428:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572546.317431:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.317435:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317455:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163d70 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317466:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317472:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317477:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572546.317479:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000519c354c reply state 000000001a03657c transno 4294970676 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.317482:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00000100:00100000:1.0:1788572546.317487:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.317576:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35938:x1875453360491136:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 633us (2311us total) trans 4294970676 rc 0/0 00000100:00100000:1.0:1788572546.317584:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4512 00010000:00010000:2.0:1788572546.318220:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.318232:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.318243:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x75d1bebc (ffff9627e7a95200) 00010000:00010000:2.0:1788572546.318245:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.318249:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.318260:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163d77 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x75d1bebc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.318295:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb163d69 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0274 expref: 92 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.318308:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163d85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0274 expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.318315:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572546.318318:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572546.318320:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc 0) 00000100:00100000:2.0:1788572546.318480:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35939:x1875453360490880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1649us (4609us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.318728:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4511 00000100:00100000:1.0:1788572546.323492:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360491264 00000100:00100000:1.0:1788572546.323508:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360491264 00000100:00100000:3.0:1788572546.323581:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.323587:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4513 00000100:00100000:1.0:1788572546.323599:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35940:x1875453360491264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.323623:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.323646:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163d8c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f027b expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.323655:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.323679:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360491392 00000100:00100000:3.0:1788572546.323684:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360491392 00000100:00100000:3.0:1788572546.323705:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360491648 00000100:00100000:3.0:1788572546.323708:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360491648 00000100:00100000:3.0:1788572546.323724:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4514 00000100:00100000:3.0:1788572546.323736:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35938:x1875453360491392:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.323745:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.323757:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0282 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.323761:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4515 00000100:00100000:2.0:1788572546.323769:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35939:x1875453360491648:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.323813:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.323815:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572546.323826:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.323832:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163d3f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f026d expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.323844:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572546.323847:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:3.0:1788572546.323861:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572546.323862:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163d85 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0274 expref: 94 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.323865:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14c600, rc 0) 00010000:00010000:2.0:1788572546.323870:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572546.323873:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572546.323888:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.323896:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.323903:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.323914:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.323917:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35938:x1875453360491392:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 186us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.323924:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4514 00010000:00010000:2.0:1788572546.323927:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.323933:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324170:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324178:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324185:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324191:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.324196:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324204:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324214:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572546.324216:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.324220:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324226:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb163d9a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324236:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324242:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324248:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324253:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.324257:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324262:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324269:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x199d9ed5 (ffff9627e7a95900) 00010000:00010000:2.0:1788572546.324271:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.324273:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.324279:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163da1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6b:0x0].0x199d9ed5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.324344:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35939:x1875453360491648:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 575us (687us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572546.324352:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4515 00010000:00010000:1.0:1788572546.324737:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.324748:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.324757:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.324841:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163d8c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.324855:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.324861:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.324864:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572546.324866:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:1.0:1788572546.324901:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35940:x1875453360491264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1305us (8236us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.324908:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4513 00010000:00010000:0.0:1788572546.327622:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.327639:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.327648:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.327655:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.327660:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.327668:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.327687:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572546.327690:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572546.327695:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.327703:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163d7e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.330778:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.330972:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360491776 00000100:00100000:2.0:1788572546.330985:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360491776 00000100:00100000:2.0:1788572546.331241:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1249 00000100:00100000:2.0:1788572546.331252:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360491776:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.332636:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360491776:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 1378us (1863us total) trans 4294967636 rc 0/0 00000100:00100000:2.0:1788572546.332649:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1249 00000100:00100000:0.0:1788572546.332835:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.332950:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.333278:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360492032 00000100:00100000:2.0:1788572546.333286:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360492032 00000100:00100000:2.0:1788572546.333371:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360492160 00000100:00100000:2.0:1788572546.333375:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360492160 00000100:00100000:3.0:1788572546.333411:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4516 00000100:00100000:3.0:1788572546.333422:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35940:x1875453360492032:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.333542:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4517 00000100:00100000:1.0:1788572546.333569:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35939:x1875453360492160:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.333580:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.333602:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163daf lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0289 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.333636:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.333649:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.333771:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.333781:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.333789:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.334071:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.334167:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163daf lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 93 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.334185:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.334205:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.334207:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572546.334210:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572546.334276:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35939:x1875453360492160:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 724us (1325us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.334283:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4517 00000040:00080000:3.0:1788572546.336319:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].70, 88 off13944 00000004:00080000:3.0:1788572546.336335:0:5837:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 894405330659ns, next at 895405327029ns, rc = 0 00000004:00010000:3.0:1788572546.336434:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f9d5c6d reply state 00000000e20dae94 transno 4294970677 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.336457:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572546.336765:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35940:x1875453360492032:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 3341us (3927us total) trans 4294970677 rc 0/0 00000100:00100000:3.0:1788572546.336775:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4516 00000100:00100000:3.0:1788572546.337676:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.337770:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360492416 00000100:00100000:0.0:1788572546.337782:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360492416 00000100:00100000:0.0:1788572546.337959:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 950 00000100:00100000:1.0:1788572546.337962:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.337971:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35938:x1875453360492416:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.338037:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360492672 00000100:00100000:3.0:1788572546.338042:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360492672 00000100:00100000:3.0:1788572546.338073:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4518 00000100:00100000:3.0:1788572546.338080:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35939:x1875453360492672:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.338168:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.338185:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.338209:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572546.338808:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].73, 88 off14136 00000004:00010000:3.0:1788572546.338897:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cc99e176 reply state 0000000087ef387a transno 4294970678 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.338908:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572546.339032:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35939:x1875453360492672:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 950us (1067us total) trans 4294970678 rc 0/0 00000100:00100000:3.0:1788572546.339041:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4518 00010000:00010000:1.0:1788572546.340933:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.340952:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.340960:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.340967:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.340971:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.340981:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.341105:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.341108:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572546.341114:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.341120:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163dbd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.341362:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35938:x1875453360492416:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 3388us (3683us total) trans 4294970679 rc 0/0 00000100:00100000:0.0:1788572546.341374:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 950 00000100:00100000:3.0:1788572546.341399:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.341954:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360492800 00000100:00100000:1.0:1788572546.341964:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360492800 00000100:00100000:1.0:1788572546.342061:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4519 00000100:00100000:1.0:1788572546.342073:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35940:x1875453360492800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.342096:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.342114:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0290 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.342171:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.342186:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.342205:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.342208:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572546.342211:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96c00, rc 0) 00000100:00100000:1.0:1788572546.342252:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35940:x1875453360492800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 183us (853us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.342262:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4519 00000100:00100000:1.0:1788572546.343389:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.343551:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360492928 00000100:00100000:3.0:1788572546.343559:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360492928 00000100:00100000:3.0:1788572546.343614:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4520 00000100:00100000:3.0:1788572546.343623:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35939:x1875453360492928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.343641:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.343808:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0297 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.343872:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.343892:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 97 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.343911:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:3.0:1788572546.343914:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.343917:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14ce00, rc 0) 00000100:00100000:3.0:1788572546.343972:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35939:x1875453360492928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 350us (596us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.343980:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4520 00000100:00100000:1.0:1788572546.344250:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.344327:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360493184 00000100:00100000:2.0:1788572546.344333:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360493184 00000100:00100000:2.0:1788572546.344381:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4521 00000100:00100000:2.0:1788572546.344390:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35938:x1875453360493184:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.344477:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35938:x1875453360493184:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 87us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.344669:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4521 00010000:00010000:1.0:1788572546.345260:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.345269:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.345274:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.345280:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.345284:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.345291:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.345305:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.345308:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572546.345312:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.345317:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163dc4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.346520:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.346696:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360493312 00000100:00100000:3.0:1788572546.346705:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360493312 00000100:00100000:3.0:1788572546.346760:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1250 00000100:00100000:3.0:1788572546.346772:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360493312:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.347121:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360493312:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 350us (605us total) trans 4294967637 rc 0/0 00000100:00100000:3.0:1788572546.347129:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1250 00000100:00100000:2.0:1788572546.349258:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.349404:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360493568 00000100:00100000:1.0:1788572546.349417:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360493568 00000100:00100000:1.0:1788572546.349518:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4522 00000100:00100000:1.0:1788572546.349530:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35938:x1875453360493568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.349551:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.349577:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163dd9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f029e expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.349649:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163de0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.349664:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163de0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.349676:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163de0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572546.349717:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572546.349741:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163de0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.349769:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163dd9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f029e expref: 98 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.349788:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163de0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f029e expref: 97 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.349796:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572546.349799:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572546.349803:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572546.349876:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35938:x1875453360493568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 349us (626us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.349887:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4522 00000100:00100000:2.0:1788572546.350924:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.351522:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360493440 00000100:00100000:1.0:1788572546.351531:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360493440 00000100:00100000:1.0:1788572546.351594:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1251 00000100:00100000:1.0:1788572546.351606:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360493440:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.351906:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360493440:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 304us (989us total) trans 4294967637 rc 0/0 00000100:00100000:1.0:1788572546.351913:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1251 00000100:00100000:3.0:1788572546.354988:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.357186:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360493824 00000100:00100000:0.0:1788572546.357272:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360493824 00000100:00100000:3.0:1788572546.357275:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.357315:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.357370:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360493952 00000100:00100000:2.0:1788572546.357430:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360493952 00000100:00100000:2.0:1788572546.357481:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1252 00000100:00100000:2.0:1788572546.357502:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360493952:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.357527:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.357555:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02a5 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572546.357588:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x54:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7381, blocks: 0x1 00010000:00010000:2.0:1788572546.357614:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=894s v=(0 0) 00010000:00010000:2.0:1788572546.357619:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.357631:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.357652:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.357666:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc 0) 00000100:00100000:2.0:1788572546.357731:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360493952:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 232us (460us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.357739:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1252 00000100:00100000:0.0:1788572546.357880:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360494208 00000100:00100000:0.0:1788572546.357885:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360494208 00000100:00100000:0.0:1788572546.357909:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 951 00000100:00100000:0.0:1788572546.357923:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35940:x1875453360493824:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.358326:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35940:x1875453360493824:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 404us (3347us total) trans 4294970680 rc 0/0 00000100:00100000:0.0:1788572546.358332:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 951 00000100:00100000:0.0:1788572546.358344:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 952 00000100:00100000:0.0:1788572546.358348:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35939:x1875453360494208:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.358774:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35939:x1875453360494208:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 422us (1454us total) trans 4294970681 rc 0/0 00000100:00100000:0.0:1788572546.358785:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 952 00000100:00100000:3.0:1788572546.362913:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.364659:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360494464 00000100:00100000:1.0:1788572546.364672:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360494464 00000100:00100000:1.0:1788572546.364747:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4523 00000100:00100000:1.0:1788572546.364757:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35940:x1875453360494464:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.364887:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35940:x1875453360494464:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 132us (1983us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.364895:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4523 00000100:00100000:3.0:1788572546.365351:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.365452:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.365585:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360494720 00000100:00100000:0.0:1788572546.365591:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360494720 00000100:00100000:0.0:1788572546.365628:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360494976 00000100:00100000:0.0:1788572546.365631:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360494976 00000100:00100000:0.0:1788572546.365653:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4524 00000100:00100000:0.0:1788572546.365660:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35939:x1875453360494720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.365729:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35939:x1875453360494720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 69us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.365737:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4524 00000100:00100000:0.0:1788572546.365752:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4525 00000100:00100000:0.0:1788572546.365758:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35938:x1875453360494976:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.365792:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35938:x1875453360494976:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 33us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.365798:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4525 00000100:00100000:3.0:1788572546.371255:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.371284:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.371347:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360495232 00000100:00100000:0.0:1788572546.371357:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360495232 00000100:00100000:0.0:1788572546.371413:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360495360 00000100:00100000:0.0:1788572546.371417:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360495360 00000100:00100000:0.0:1788572546.371453:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4526 00000100:00100000:0.0:1788572546.371464:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35939:x1875453360495232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.371552:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.371580:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163dee lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02b3 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371661:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb163df5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371676:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb163df5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371687:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb163df5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.371728:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.371749:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb163df5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371776:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb163dee lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02b3 expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371797:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb163df5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02b3 expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371807:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:0.0:1788572546.371810:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572546.371814:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bda00, rc 0) 00000100:00100000:0.0:1788572546.371870:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35939:x1875453360495232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 408us (625us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.371878:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4526 00000100:00100000:0.0:1788572546.371898:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4527 00000100:00100000:0.0:1788572546.371905:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35940:x1875453360495360:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.371916:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.371926:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163dfc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02ba expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371954:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163e03 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371964:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163e03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.371972:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163e03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.371988:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.371998:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163e03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.372039:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb163dfc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ba expref: 102 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.372052:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163e03 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ba expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.372059:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:0.0:1788572546.372062:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572546.372065:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc200, rc 0) 00000100:00100000:0.0:1788572546.372084:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35940:x1875453360495360:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 180us (799us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.372091:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4527 00000100:00100000:0.0:1788572546.379228:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.379430:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.384173:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.387700:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360495104 00000100:00100000:3.0:1788572546.387719:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360495104 00000100:00100000:3.0:1788572546.387812:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360495616 00000100:00100000:3.0:1788572546.387816:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360495616 00000100:00100000:3.0:1788572546.387853:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1253 00000100:00100000:3.0:1788572546.387863:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35938:x1875453360495104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.387882:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.387902:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02ac expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.387936:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 14 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.387945:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.387949:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.387954:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.387991:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 14 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.387994:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1254 00000100:00100000:2.0:1788572546.388017:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360495616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.388019:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 14 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.388026:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788572546.388034:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382378496:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572546.388039:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02c1 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572546.388062:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x54:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7381, blocks: 0x1 00010000:00010000:2.0:1788572546.388074:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=894s v=(0 0) 00010000:00010000:2.0:1788572546.388077:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.388084:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 13 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.388100:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 13 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.388113:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e800, rc 0) 00000100:00100000:2.0:1788572546.388150:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360495616:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 148us (8721us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.388157:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1254 00000100:00100000:3.0:1788572546.388228:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.388953:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360495488 00000100:00100000:2.0:1788572546.388960:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360495488 00000100:00100000:2.0:1788572546.389071:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1255 00000100:00100000:2.0:1788572546.389079:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360495488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.389090:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.389105:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02c8 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572546.389126:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x55:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7381, blocks: 0x1 00010000:00010000:2.0:1788572546.389138:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=894s v=(0 0) 00010000:00010000:2.0:1788572546.389142:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.389148:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.389167:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 16 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.389178:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e200, rc 0) 00000100:00100000:2.0:1788572546.389223:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360495488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 144us (5063us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.389230:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1255 00000100:00100000:3.0:1788572546.395676:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.395810:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360495872 00000100:00100000:1.0:1788572546.395815:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382378496:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572546.395819:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360495872 00010000:00010000:1.0:1788572546.395848:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 15 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.395870:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d200, rc 0) 00000100:00100000:2.0:1788572546.395887:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4528 00000100:00100000:2.0:1788572546.395896:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35940:x1875453360495872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.400046:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35940:x1875453360495872:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 4136us (4375us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.400064:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4528 00000100:00100000:0.0:1788572546.404239:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.404320:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.404407:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35938:x1875453360495104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 16541us (25186us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.404419:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1253 00000100:00100000:2.0:1788572546.404421:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496000 00000100:00100000:2.0:1788572546.404432:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496000 00010000:00010000:2.0:1788572546.404484:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 15 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.404519:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1068 00000100:00100000:2.0:1788572546.404528:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360496000:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.404548:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.404576:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 15 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.404592:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 15 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.404617:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0580b65/0x85ace67feb163de7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02a5 expref: 15 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.404634:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.404639:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.404649:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.404746:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.404795:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.404811:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382378624:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572546.409428:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.413621:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496256 00000100:00100000:1.0:1788572546.413637:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496256 00000100:00100000:1.0:1788572546.413741:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4529 00000100:00100000:1.0:1788572546.413755:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35939:x1875453360496256:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.417349:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.417493:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.417944:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496512 00000100:00100000:2.0:1788572546.417954:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496512 00002000:00100000:2.0:1788572546.418043:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x54:0x0] (0->2) req@ffff9627c84efb80 x1875453360496512/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572546.418056:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84efb80 with x1875453360496512 ext(0->2) 00002000:00010000:2.0:1788572546.418063:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84efb80 00000100:00100000:2.0:1788572546.418088:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 282 00000100:00100000:2.0:1788572546.418097:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360496512:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.418569:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2349:x1875453360496512:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 472us (1074us total) trans 4294967638 rc 0/0 00000100:00100000:2.0:1788572546.418590:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 282 00002000:00100000:2.0:1788572546.418598:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x54:0x0] (0->2) req@ffff9627c84efb80 x1875453360496512/t4294967638(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572546.418606:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84efb80 with x1875453360496512 ext(0->2) 00002000:00010000:2.0:1788572546.418609:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84efb80 00000100:00100000:2.0:1788572546.418636:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496384 00000100:00100000:2.0:1788572546.418641:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496384 00000100:00100000:2.0:1788572546.418750:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1256 00000100:00100000:2.0:1788572546.418759:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35940:x1875453360496384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.418769:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.418786:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02cf expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.418813:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 14 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.418822:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.418891:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.418898:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.418937:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 14 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.418951:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 14 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.418964:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382378752:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572546.419041:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f240 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.419082:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:11655:1875453382378624:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572546.419101:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.419122:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360496000:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 14596us (14894us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.419130:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1068 00000100:00100000:0.0:1788572546.422324:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.422407:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496640 00000100:00100000:1.0:1788572546.422416:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496640 00000100:00100000:3.0:1788572546.422467:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382378752:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.422478:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 15 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.422508:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.422517:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1069 00010000:00010000:3.0:1788572546.422527:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc 0) 00000100:00100000:1.0:1788572546.422528:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9da850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360496640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.422545:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.422568:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 15 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.422646:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 15 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.422741:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071068e57/0x85ace67feb163e11 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c1 expref: 15 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.422758:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.422763:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.422772:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.422783:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.422821:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.422837:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382378880:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572546.427332:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35940:x1875453360496384:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8560us (9980us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.427347:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1256 00000100:00100000:2.0:1788572546.430453:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35939:x1875453360496256:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 16679us (26109us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.430470:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4529 00000100:00100000:0.0:1788572546.430585:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.432519:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.432599:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496768 00000100:00100000:1.0:1788572546.432610:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496768 00000100:00100000:1.0:1788572546.432753:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4530 00000100:00100000:1.0:1788572546.432769:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35938:x1875453360496768:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.440406:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382378880:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572546.440463:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.440531:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9da850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360496640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 18005us (18217us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.440544:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1069 00000100:00100000:2.0:1788572546.443742:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.443782:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.443809:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360496896 00000100:00100000:3.0:1788572546.443819:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360496896 00000100:00100000:1.0:1788572546.443856:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497024 00000100:00100000:1.0:1788572546.443861:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497024 00000100:00100000:3.0:1788572546.443920:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1257 00002000:00100000:1.0:1788572546.443925:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x54:0x0] (0->2) req@ffff9627dfae5c00 x1875453360497024/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1788572546.443931:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35939:x1875453360496896:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00002000:00100000:1.0:1788572546.443937:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae5c00 with x1875453360497024 ext(0->2) 00002000:00010000:1.0:1788572546.443942:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfae5c00 00010000:00010000:3.0:1788572546.443946:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788572546.443963:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 283 00010000:00010000:3.0:1788572546.443964:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02d6 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.443972:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360497024:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.443995:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.444026:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.444030:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.444036:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 17 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.444069:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.444087:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.444114:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382379008:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.444188:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.444410:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360497024:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 439us (625us total) trans 4294967638 rc 0/0 00000100:00100000:1.0:1788572546.444418:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 283 00002000:00100000:1.0:1788572546.444425:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x54:0x0] (0->2) req@ffff9627dfae5c00 x1875453360497024/t4294967638(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.444434:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae5c00 with x1875453360497024 ext(0->2) 00002000:00010000:1.0:1788572546.444448:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfae5c00 00000100:00100000:3.0:1788572546.448598:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.448712:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.448965:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497152 00000100:00100000:1.0:1788572546.448975:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497152 00000100:00100000:1.0:1788572546.449075:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4531 00000100:00100000:1.0:1788572546.449087:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35940:x1875453360497152:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.449666:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382379008:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572546.449698:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 17 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.449720:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14cc00, rc 0) 00000100:00100000:2.0:1788572546.449840:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35939:x1875453360496896:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 5909us (6104us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.449985:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1257 00000100:00100000:2.0:1788572546.450050:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497280 00000100:00100000:2.0:1788572546.450055:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497280 00010000:00010000:2.0:1788572546.450080:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.450106:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1070 00000100:00100000:2.0:1788572546.450114:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360497280:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.450126:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.450148:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.450161:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.450185:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb163e18 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02c8 expref: 17 pid: 20911 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.450215:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.450219:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.450227:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.450237:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.450272:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 16 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.450286:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382379136:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572546.450340:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.457715:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.457801:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497408 00000100:00100000:1.0:1788572546.457812:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497408 00000100:00100000:3.0:1788572546.457906:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382379136:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572546.457934:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.458001:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360497280:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7886us (9287us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.458037:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1070 00002000:00100000:1.0:1788572546.458391:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x55:0x0] (0->2) req@ffff9627c84ef800 x1875453360497408/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.458403:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ef800 with x1875453360497408 ext(0->2) 00002000:00010000:1.0:1788572546.458408:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84ef800 00000100:00100000:1.0:1788572546.458454:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 284 00000100:00100000:1.0:1788572546.458467:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360497408:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.459045:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360497408:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 580us (1338us total) trans 4294967639 rc 0/0 00000100:00100000:1.0:1788572546.459053:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 284 00002000:00100000:1.0:1788572546.459059:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x55:0x0] (0->2) req@ffff9627c84ef800 x1875453360497408/t4294967639(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.459067:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ef800 with x1875453360497408 ext(0->2) 00002000:00010000:1.0:1788572546.459070:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84ef800 00000100:00100000:0.0:1788572546.471168:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.471209:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497536 00000100:00100000:0.0:1788572546.471218:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497536 00000100:00100000:0.0:1788572546.471270:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4532 00000100:00100000:0.0:1788572546.471280:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35939:x1875453360497536:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1788572546.499167:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970681 is committed 00000020:00080000:2.0:1788572546.499200:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d13170c0 refcount 2 committed 0 result 0 batchid 131 00000020:00080000:2.0:1788572546.499206:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aea00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d13176c0 00000020:00080000:2.0:1788572546.499210:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9628009de400 00000020:00080000:2.0:1788572546.499224:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2f64480 refcount 2 committed 0 result 0 batchid 130 00000020:00080000:2.0:1788572546.499227:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae700 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c9bda180 00000020:00080000:2.0:1788572546.499231:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aef00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627e7426400 00000020:00080000:2.0:1788572546.499241:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 129 batchid updated 00000004:00080000:2.0:1788572546.499253:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 44 committed 00000004:00080000:2.0:1788572546.499295:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 39 committed 00000020:00080000:2.0:1788572546.499312:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2463000 refcount 2 committed 0 result 0 batchid 132 00000020:00080000:2.0:1788572546.499316:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceac00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627f582af00 00000020:00080000:2.0:1788572546.499318:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cea300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f5977000 00000100:00100000:2.0:1788572546.499535:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35940:x1875453360497152:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 50448us (50943us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.499547:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4531 00000100:00100000:0.0:1788572546.499643:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35939:x1875453360497536:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 27633us (27754us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.499651:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4532 00000020:00080000:0.0:1788572546.499802:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 132 committed batchid 129 00000020:00080000:0.0:1788572546.500030:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 129 00000100:00100000:0.0:1788572546.500125:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382379648:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572546.500181:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572546.500214:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788572546.500220:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382379520:0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572546.500242:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572546.500272:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382379648 00000100:00100000:0.0:1788572546.500279:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382379648 00000100:00100000:0.0:1788572546.500316:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382379520 00000100:00100000:0.0:1788572546.500320:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382379520 00000100:00100000:0.0:1788572546.500339:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1258 00000100:00100000:0.0:1788572546.500347:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3305:x1875453382379648:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572546.500524:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382379648:12345-0@lo:2:osp-syn-1-0.0 Request processed in 177us (342us total) trans 4294967639 rc 0/0 00000100:00100000:0.0:1788572546.500532:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1258 00000100:00100000:0.0:1788572546.500546:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1259 00000100:00100000:0.0:1788572546.500553:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3305:x1875453382379520:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:0.0:1788572546.500641:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3305:x1875453382379520:12345-0@lo:2:osp-syn-1-0.0 Request processed in 88us (398us total) trans 4294967640 rc 0/0 00000100:00100000:0.0:1788572546.500647:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1259 00000004:00080000:0.0:1788572546.500694:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c84eea00/2, rc 0, transno 4294967640 00000100:00100000:0.0:1788572546.500701:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382379520:0@lo:2:osp-syn-1-0.0 00000004:00080000:0.0:1788572546.500709:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c84ec000/2, rc 0, transno 4294967639 00000100:00100000:0.0:1788572546.500712:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382379648:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572546.501128:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382379392:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572546.501167:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572546.501205:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572546.501212:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382379264:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572546.501237:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572546.501264:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382379776:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572546.501283:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572546.501371:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35938:x1875453360496768:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 68606us (68860us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.501380:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4530 00000100:00100000:2.0:1788572546.501459:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382379392 00000100:00100000:2.0:1788572546.501465:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382379392 00000100:00100000:2.0:1788572546.501499:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382379264 00000100:00100000:2.0:1788572546.501501:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382379264 00000100:00100000:2.0:1788572546.501515:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382379776 00000100:00100000:2.0:1788572546.501518:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382379776 00000100:00100000:2.0:1788572546.501533:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1260 00000100:00100000:2.0:1788572546.501540:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3303:x1875453382379392:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572546.501653:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572546.501658:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb mtime from disk: 0 -> 1788572545 00002000:00010000:2.0:1788572546.501660:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1788572546.501662:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb ctime from disk: 1788572545 -> 1788572545 00002000:00010000:2.0:1788572546.501664:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1788572546.501700:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382379392:12345-0@lo:2:osp-syn-0-0.0 Request processed in 161us (533us total) trans 4294967640 rc 0/0 00000100:00100000:2.0:1788572546.501707:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1260 00000100:00100000:2.0:1788572546.501722:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1261 00000100:00100000:2.0:1788572546.501729:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3303:x1875453382379264:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572546.501818:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382379264:12345-0@lo:2:osp-syn-0-0.0 Request processed in 89us (579us total) trans 4294967641 rc 0/0 00000100:00100000:2.0:1788572546.501824:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1261 00000100:00100000:2.0:1788572546.501837:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1262 00000100:00100000:2.0:1788572546.501844:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382379776:12345-0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1788572546.503093:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfae7100/2, rc 0, transno 4294967641 00000100:00100000:1.0:1788572546.503112:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382379264:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1788572546.503124:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627dfae5880/2, rc 0, transno 4294967640 00000100:00100000:1.0:1788572546.503127:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382379392:0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572546.503991:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572546.503997:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb mtime from disk: 0 -> 1788572545 00002000:00010000:2.0:1788572546.504000:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1788572546.504052:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb ctime from disk: 1788572545 -> 1788572545 00002000:00010000:2.0:1788572546.504056:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1788572546.504174:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+10:3303:x1875453382379776:12345-0@lo:2:osp-syn-1-0.0 Request processed in 2328us (2887us total) trans 4294967641 rc 0/0 00000100:00100000:2.0:1788572546.504183:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1262 00000100:00100000:0.0:1788572546.504322:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.504430:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.504467:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497664 00000100:00100000:0.0:1788572546.504476:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497664 00000100:00100000:0.0:1788572546.504510:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497792 00000100:00100000:0.0:1788572546.504513:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497792 00000100:00100000:0.0:1788572546.504529:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1263 00000100:00100000:0.0:1788572546.504538:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360497664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.504632:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1264 00000100:00100000:0.0:1788572546.504640:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360497792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000004:00080000:1.0:1788572546.514367:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c84ef480/2, rc 0, transno 4294967641 00000100:00100000:1.0:1788572546.514382:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382379776:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572546.515085:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.517541:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360497920 00000100:00100000:0.0:1788572546.517554:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360497920 00000100:00100000:0.0:1788572546.517616:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1265 00000100:00100000:0.0:1788572546.517628:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2347:x1875453360497920:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.517749:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2347:x1875453360497920:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 123us (2668us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.517757:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1265 00000100:00100000:1.0:1788572546.532366:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.532495:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360498176 00000100:00100000:1.0:1788572546.532505:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360498176 00000100:00100000:1.0:1788572546.532593:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4533 00000100:00100000:1.0:1788572546.532606:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35938:x1875453360498176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.532665:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.532695:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163de0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f029e expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.532714:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572546.532719:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572546.532750:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.532768:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.532779:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572546.533161:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000976e18e0 reply state 000000000bb26173 transno 4294970682 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.533172:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.533317:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35938:x1875453360498176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 713us (962us total) trans 4294970682 rc 0/0 00000100:00100000:1.0:1788572546.533325:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4533 00000100:00100000:1.0:1788572546.537715:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572546.537748:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.537762:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.537771:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.537778:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.537783:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.537793:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.537818:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572546.537822:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.537827:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.537833:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e2d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.538360:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360498304 00000100:00100000:2.0:1788572546.538370:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360498304 00000100:00100000:2.0:1788572546.538415:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1266 00000100:00100000:2.0:1788572546.538426:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2349:x1875453360498304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572546.538680:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572546.538684:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb mtime from disk: 1788572545 -> 1788572521 00002000:00010000:2.0:1788572546.538687:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:2.0:1788572546.538689:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb ctime from disk: 1788572545 -> 1788572523 00002000:00010000:2.0:1788572546.538691:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x54:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572546.538748:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+20:2349:x1875453360498304:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 324us (1046us total) trans 4294967642 rc 0/0 00000100:00100000:2.0:1788572546.538757:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1266 00000100:00100000:1.0:1788572546.541749:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.542471:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360498560 00000100:00100000:2.0:1788572546.542481:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360498560 00000100:00100000:2.0:1788572546.542545:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4534 00000100:00100000:2.0:1788572546.542556:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35938:x1875453360498560:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.542629:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.542646:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.542656:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.542695:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.542709:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 65 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.542722:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382379904:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572546.542805:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3180 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.547357:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382379904:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.547395:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.553539:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.553623:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360498688 00000100:00100000:3.0:1788572546.553633:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360498688 00010000:00010000:3.0:1788572546.553680:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 66 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.553726:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1071 00000100:00100000:3.0:1788572546.553738:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360498688:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.553754:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.553762:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 66 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.553779:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 66 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.553804:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163d54 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x99:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0266 expref: 66 pid: 23058 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.553820:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:3.0:1788572546.553826:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.553838:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.553846:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.553871:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.553930:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360498688:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 194us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.553938:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1071 00010000:00010000:1.0:1788572546.555662:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555678:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555710:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555718:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555725:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555731:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.555736:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555744:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555759:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:1.0:1788572546.555763:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572546.555767:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.555772:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163e34 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.555861:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35938:x1875453360498560:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13302us (14117us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572546.555871:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4534 00000100:00100000:1.0:1788572546.560643:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.564781:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360498816 00000100:00100000:0.0:1788572546.564796:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360498816 00000100:00100000:0.0:1788572546.564854:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1267 00000100:00100000:0.0:1788572546.564866:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7663450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+20:2347:x1875453360498816:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.564921:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.564944:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 20 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.564953:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.564957:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572546.564962:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 4/1,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.564998:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 20 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.565035:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 20 pid: 10416 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.565052:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382380032:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572546.565137:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b79a80 going to sleep for 11 seconds 00000001:00080000:2.0:1788572546.575735:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967634 is committed 00000001:00080000:2.0:1788572546.575759:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967639 is committed 00000001:00080000:2.0:1788572546.575772:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967641 is committed 00000100:00100000:0.0:1788572546.576628:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.576705:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360498944 00000100:00100000:3.0:1788572546.576714:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360498944 00010000:00010000:3.0:1788572546.576750:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.576792:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1072 00000100:00100000:3.0:1788572546.576801:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360498944:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.576817:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.576838:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.576853:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 14 pid: 10416 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.576884:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.576889:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.576896:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.576908:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.576938:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.576982:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360498944:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 182us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.576990:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1072 00010000:00010000:2.0:1788572546.577137:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb163e0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ac expref: 13 pid: 10416 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.577153:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382380032:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572546.577177:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577203:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577212:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577238:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577245:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577253:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577260:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572546.577265:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577274:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577286:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.577299:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb163e3b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.577341:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7663450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360498816:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 12479us (16708us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.577348:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1267 00000100:00100000:1.0:1788572546.578034:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360497664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 73492us (73709us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.578043:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1263 00000100:00100000:1.0:1788572546.581100:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.581135:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.581167:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360499200 00000100:00100000:1.0:1788572546.581176:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360499200 00000100:00100000:1.0:1788572546.581251:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360499456 00000100:00100000:1.0:1788572546.581254:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360499456 00000100:00100000:1.0:1788572546.581277:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4535 00000100:00100000:1.0:1788572546.581287:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35939:x1875453360499200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.581351:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.581379:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb163df5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02b3 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.581398:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.581403:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572546.581436:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.581460:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.581470:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572546.581650:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000706cfa3f reply state 000000005e5dfea3 transno 4294970683 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.581659:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.581760:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35939:x1875453360499200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 473us (668us total) trans 4294970683 rc 0/0 00000100:00100000:1.0:1788572546.581768:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4535 00000100:00100000:1.0:1788572546.581789:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4536 00000100:00100000:1.0:1788572546.581800:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35938:x1875453360499456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.581842:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.581852:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.581863:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.582038:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.582049:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.582057:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.582091:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.582106:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.582122:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382380160:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572546.582176:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9fae40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.587969:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.588090:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360499584 00000100:00100000:3.0:1788572546.588100:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360499584 00000100:00100000:3.0:1788572546.588152:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1268 00000100:00100000:3.0:1788572546.588163:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360499584:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572546.588398:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x55:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572546.588402:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x55:0x0] updating lvb mtime from disk: 0 -> 1788572521 00002000:00010000:3.0:1788572546.588405:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x55:0x0] updating lvb atime from disk: 0 -> 1788572521 00002000:00010000:3.0:1788572546.588407:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x55:0x0] updating lvb ctime from disk: 1788572545 -> 1788572524 00002000:00010000:3.0:1788572546.588409:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x55:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572546.588474:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360499584:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 312us (514us total) trans 4294967643 rc 0/0 00000100:00100000:3.0:1788572546.588483:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1268 00010000:00010000:1.0:1788572546.588688:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.588698:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.588706:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.588717:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.588723:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.588730:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.588746:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.588749:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.588754:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.588761:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb163e42 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.588877:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382380160:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.588912:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.591147:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.591296:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360499712 00000100:00100000:3.0:1788572546.591306:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360499712 00010000:00010000:3.0:1788572546.591341:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.591385:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1073 00000100:00100000:3.0:1788572546.591395:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360499712:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.591408:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.591416:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.591429:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.591461:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163d93 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0282 expref: 66 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.591477:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:3.0:1788572546.591482:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.591492:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.591500:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.591526:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.591579:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360499712:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 184us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.591586:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1073 00010000:00010000:1.0:1788572546.592098:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.592107:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572546.592498:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970684 00000001:00080000:1.0:1788572546.592666:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970685 00000001:00080000:1.0:1788572546.592728:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970686 00000100:00100000:1.0:1788572546.592827:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382380288:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572546.592918:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572546.593102:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000001:00080000:1.0:1788572546.594296:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967640 is committed 00000001:00080000:1.0:1788572546.594317:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967634 is committed 00000001:00080000:1.0:1788572546.594325:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967638 is committed 00000001:00080000:1.0:1788572546.594333:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967641 is committed 00000100:00100000:1.0:1788572546.595405:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360497792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 90758us (90967us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.595417:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1264 00000100:00100000:0.0:1788572546.596646:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382380288 00000100:00100000:0.0:1788572546.596658:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382380288 00000100:00100000:0.0:1788572546.596715:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:0.0:1788572546.596725:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382380288:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572546.597162:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382380288:12345-0@lo:1000:mdt00_003.0 Request processed in 437us (4243us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.597170:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1788572546.599186:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.599272:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360499968 00000100:00100000:1.0:1788572546.599281:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360499968 00000100:00100000:1.0:1788572546.599345:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4537 00000100:00100000:1.0:1788572546.599355:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35940:x1875453360499968:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.599413:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.599447:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb163e03 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02ba expref: 68 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.599465:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.599469:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572546.599499:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.599513:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.599524:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572546.599683:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c655f382 reply state 0000000045631161 transno 4294970688 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.599697:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.599808:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35940:x1875453360499968:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 454us (629us total) trans 4294970688 rc 0/0 00000100:00100000:1.0:1788572546.599815:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4537 00000100:00100000:3.0:1788572546.600631:0:9040:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967641 gen 2 00000004:00080000:3.0:1788572546.600638:0:9040:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967635 committed transno 4294967641 00000004:00080000:3.0:1788572546.600658:0:9040:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfae5880, transno 4294967640 00000004:00080000:3.0:1788572546.600672:0:9040:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627dfae7100, transno 4294967641 00000100:00100000:3.0:1788572546.600681:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382380288:0@lo:1000:mdt00_003.0 00000001:00080000:3.0:1788572546.601151:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970687 00000100:00100000:3.0:1788572546.601767:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ebb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382380416:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572546.601830:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572546.601855:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c63cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.602226:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382380416 00000100:00100000:0.0:1788572546.602234:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382380416 00000100:00100000:0.0:1788572546.602285:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1788572546.602293:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c97cec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:9040:x1875453382380416:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572546.602503:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c97cec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382380416:12345-0@lo:1000:mdt00_003.0 Request processed in 210us (671us total) trans 4294967644 rc 0/0 00000100:00100000:0.0:1788572546.602510:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:3.0:1788572546.602808:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ebb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382380416:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1788572546.602858:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602870:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602879:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602885:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.602890:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602899:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602927:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:3.0:1788572546.602931:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.602936:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602942:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb163e50 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x99:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602976:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602982:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602987:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.602993:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.602996:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603033:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603043:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627d55e6200) 00010000:00010000:3.0:1788572546.603046:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.603050:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603060:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb15fb60 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x71:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603083:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603095:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603105:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603112:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603118:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603124:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603129:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.603131:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603138:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603145:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627d55e6200) 00010000:00010000:3.0:1788572546.603148:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.603151:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603157:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163e5e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x71:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603165:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603171:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603176:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603180:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.603183:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603188:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603195:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627d55e6200) 00010000:00010000:3.0:1788572546.603198:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572546.603201:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.603206:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb163e49 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x71:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.603388:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35938:x1875453360499456:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 21590us (22249us total) trans 4294970689 rc 0/0 00000100:00100000:3.0:1788572546.603398:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4536 00000020:00080000:3.0:1788572546.603402:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00010000:00010000:3.0:1788572546.607629:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.607647:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.607656:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.607661:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.607666:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.607675:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.607693:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:3.0:1788572546.607696:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.607700:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.607707:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb163e57 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.608582:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.608669:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.608702:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360500608 00000100:00100000:0.0:1788572546.608713:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360500608 00000100:00100000:0.0:1788572546.608773:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 953 00000100:00100000:0.0:1788572546.608784:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35938:x1875453360500608:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572546.610055:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x99:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572546.610157:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970690 00000100:00100000:3.0:1788572546.610350:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360500352 00000100:00100000:3.0:1788572546.610357:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360500352 00000100:00100000:3.0:1788572546.610409:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1269 00000100:00100000:3.0:1788572546.610420:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360500352:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572546.610606:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:3.0:1788572546.610609:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb mtime from disk: 1788572545 -> 1788572522 00002000:00010000:3.0:1788572546.610612:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb atime from disk: 0 -> 1788572522 00002000:00010000:3.0:1788572546.610614:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb ctime from disk: 1788572545 -> 1788572523 00002000:00010000:3.0:1788572546.610616:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x54:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572546.610679:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360500352:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 260us (2104us total) trans 4294967642 rc 0/0 00000100:00100000:3.0:1788572546.610685:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1269 00000100:00100000:1.0:1788572546.613316:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.613466:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360500224 00000100:00100000:3.0:1788572546.613477:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360500224 00000100:00100000:3.0:1788572546.613536:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4538 00000100:00100000:3.0:1788572546.613546:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35939:x1875453360500224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.613616:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.613633:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 71 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.613643:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.613679:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 71 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.613694:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 71 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.613708:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382380544:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.613793:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13176c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572546.615141:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.615160:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.615173:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.615181:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.615194:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.615201:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.615217:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572546.615337:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35938:x1875453360500608:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 6550us (6663us total) trans 4294970691 rc 0/0 00000100:00100000:1.0:1788572546.615346:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 953 00000020:00080000:1.0:1788572546.615351:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572546.616418:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572546.616428:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.616436:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572546.616451:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.616461:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.616475:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x99:0x0].0x0 (ffff9627c8ed8000) 00010000:00010000:0.0:1788572546.616478:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572546.616483:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.616489:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb163e6c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x99:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.616511:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572546.616513:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572546.616515:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572546.618978:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.619569:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360500864 00000100:00100000:3.0:1788572546.619581:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360500864 00000100:00100000:3.0:1788572546.619639:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4539 00000100:00100000:3.0:1788572546.619650:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35940:x1875453360500864:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.619731:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.619747:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.619756:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.619791:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.619804:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 71 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.619819:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382380672:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.619927:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.621592:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382380544:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.621619:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.621921:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.622537:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360500992 00000100:00100000:1.0:1788572546.622545:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360500992 00010000:00010000:1.0:1788572546.622601:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 72 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.622640:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1074 00000100:00100000:1.0:1788572546.622652:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2350:x1875453360500992:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.622666:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.622672:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 72 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.622685:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 72 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.622707:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163db6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0289 expref: 72 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.622721:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.622726:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.622736:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.622742:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.622777:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.622832:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2350:x1875453360500992:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 183us (911us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.622839:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1074 00010000:00010000:2.0:1788572546.623312:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623321:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623351:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623358:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623364:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623369:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.623374:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623381:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623391:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:2.0:1788572546.623393:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572546.623397:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.623403:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163e65 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.623470:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35939:x1875453360500224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 9926us (10163us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572546.623479:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4538 00000100:00100000:3.0:1788572546.631544:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.631588:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.631727:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360501120 00000100:00100000:2.0:1788572546.631738:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360501120 00010000:00010000:2.0:1788572546.631787:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 70 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.631833:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1075 00000100:00100000:2.0:1788572546.631843:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360501120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.631869:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.631877:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 70 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.631891:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 70 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.631925:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:2.0:1788572546.631931:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.631942:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.631950:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.631972:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.632305:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360501120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 460us (771us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.632313:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1075 00010000:00010000:1.0:1788572546.632567:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163da8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f027b expref: 69 pid: 9040 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.632587:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382380672:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.632608:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632618:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632623:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632656:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632664:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632671:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632677:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.632682:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632694:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632705:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.632708:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572546.632711:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.632718:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163e73 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.632770:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35940:x1875453360500864:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 13122us (13801us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572546.632778:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4539 00000100:00100000:1.0:1788572546.632829:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360501248 00000100:00100000:1.0:1788572546.632834:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360501248 00000100:00100000:1.0:1788572546.633209:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1270 00000100:00100000:1.0:1788572546.633222:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360501248:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.633258:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.633275:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.633282:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.633286:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.633291:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 4/1,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.633323:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.633337:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 12 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.633349:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382380800:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572546.633399:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.636544:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.636613:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382380800:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.636655:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.639458:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.639624:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360501504 00000100:00100000:1.0:1788572546.639635:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360501504 00010000:00010000:1.0:1788572546.639692:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 13 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.639792:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360501376 00000100:00100000:3.0:1788572546.639797:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360501376 00000100:00100000:3.0:1788572546.639837:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1271 00000100:00100000:3.0:1788572546.639860:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360501376:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.639906:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.639933:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.639942:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572546.639946:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572546.639950:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.639989:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.640017:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 11 pid: 6679 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.640032:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382380928:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.640112:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.640337:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1076 00000100:00100000:1.0:1788572546.640350:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9db850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360501504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.640362:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.640386:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 13 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.640401:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 13 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.640421:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043bff4d5/0x85ace67feb163e26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02d6 expref: 13 pid: 13272 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.640437:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.640449:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.640455:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.640464:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.640490:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.640526:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9db850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360501504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 181us (1086us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.640533:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1076 00010000:00010000:0.0:1788572546.644334:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644355:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644395:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644409:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644418:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644425:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572546.644429:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644448:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644469:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.644477:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093fd62da/0x85ace67feb163e7a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.644593:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360501248:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11371us (13000us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.644604:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1270 00000100:00100000:3.0:1788572546.647413:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.647462:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.647494:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360501888 00000100:00100000:3.0:1788572546.647503:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360501888 00000100:00100000:3.0:1788572546.647553:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4540 00000100:00100000:3.0:1788572546.647563:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35939:x1875453360501888:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.647651:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.647670:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.647681:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.647941:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.647954:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 69 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.647963:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 4/0,1 mode: --/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.648000:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 69 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.648041:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 69 pid: 5838 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.648055:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382381056:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.648126:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d160c480 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.648366:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382380928:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572546.648389:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.648857:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360501632 00000100:00100000:1.0:1788572546.648865:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360501632 00010000:00010000:1.0:1788572546.648926:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 12 pid: 6679 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.648961:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1077 00000100:00100000:1.0:1788572546.648972:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360501632:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.648983:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.649032:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 12 pid: 6679 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.649050:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 12 pid: 6679 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.649069:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5326c2a/0x85ace67feb163e1f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x54:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02cf expref: 12 pid: 6679 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.649086:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.649091:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572546.649098:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.649108:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.649133:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.649179:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360501632:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 210us (1776us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.649187:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1077 00010000:00010000:2.0:1788572546.649631:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649642:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649668:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649677:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649686:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649693:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572546.649697:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649706:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649720:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.649727:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000526e7136/0x85ace67feb163e81 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.649783:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360501376:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 9936us (13249us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.649791:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1271 00000100:00100000:3.0:1788572546.653295:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.653383:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360502144 00000100:00100000:1.0:1788572546.653394:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360502144 00000100:00100000:1.0:1788572546.653475:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4541 00000100:00100000:1.0:1788572546.653487:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35940:x1875453360502144:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.653576:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.653594:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.653607:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.653850:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.653861:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 71 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.653869:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 4/0,1 mode: --/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.654156:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 71 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.654174:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 71 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.654191:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382381184:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572546.656703:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382381056:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572546.656731:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.657296:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c33f7300 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.658869:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.658947:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360502272 00000100:00100000:2.0:1788572546.658956:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360502272 00010000:00010000:2.0:1788572546.659000:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 72 pid: 5838 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.659089:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1078 00000100:00100000:2.0:1788572546.659100:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2349:x1875453360502272:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.659117:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.659124:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 72 pid: 5838 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.659136:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 72 pid: 5838 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.659158:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163dd2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0297 expref: 72 pid: 5838 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.659176:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:2.0:1788572546.659180:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.659201:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.659209:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.659240:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572546.659287:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2349:x1875453360502272:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 189us (427us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.659295:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1078 00010000:00010000:3.0:1788572546.659452:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.659460:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572546.659667:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970692 00000001:00080000:3.0:1788572546.659838:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970694 00000001:00080000:3.0:1788572546.659891:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970695 00000100:00100000:3.0:1788572546.659982:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382381312:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572546.660087:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572546.660118:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3780 going to sleep for 11 seconds 00000100:00100000:2.0:1788572546.660361:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382381312 00000100:00100000:2.0:1788572546.660368:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382381312 00000100:00100000:2.0:1788572546.660411:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:2.0:1788572546.660420:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382381312:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572546.660509:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382381312:12345-0@lo:1000:mdt00_002.0 Request processed in 90us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.660516:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:3.0:1788572546.661416:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382381312:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1788572546.661815:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970696 00000100:00100000:3.0:1788572546.661902:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12eb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382381440:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572546.662116:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572546.662205:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb33c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.664910:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382381440 00000100:00100000:1.0:1788572546.664928:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382381440 00000100:00100000:3.0:1788572546.664988:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.665019:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1788572546.665032:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382381440:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572546.665221:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382381184:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572546.665256:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.665422:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5839:x1875453382381440:12345-0@lo:1000:mdt00_002.0 Request processed in 393us (3305us total) trans 4294967645 rc 0/0 00000100:00100000:1.0:1788572546.665431:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1788572546.665466:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360502400 00000100:00100000:3.0:1788572546.665469:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12eb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382381440:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1788572546.665470:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360502400 00010000:00010000:3.0:1788572546.665509:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665517:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665523:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665533:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.665538:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665547:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665563:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627c3da6800) 00010000:00010000:1.0:1788572546.665564:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 75 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665566:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.665570:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665577:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163e8f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665609:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665615:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665620:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665626:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962800935a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.665629:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665635:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665647:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc16d00) 00010000:00010000:3.0:1788572546.665649:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.665653:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 2/0,0 mode: --/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665658:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962800935a00/0x85ace67feb15fbbb lrc: 0/0,0 mode: --/EX res: [0x200000401:0x74:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.665667:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1079 00000100:00100000:1.0:1788572546.665677:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360502400:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.665678:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665690:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.665690:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665695:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 75 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665698:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665705:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665706:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 75 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665712:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665717:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665721:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.665723:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163dcb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0290 expref: 75 pid: 5839 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665724:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665730:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665734:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:1.0:1788572546.665738:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665738:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc16d00) 00010000:00010000:3.0:1788572546.665740:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.665743:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665746:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665749:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163ea4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x74:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665753:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665758:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665764:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665769:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665774:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.665777:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.665778:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572546.665783:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665789:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc16d00) 00010000:00010000:3.0:1788572546.665791:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572546.665794:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.665799:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163e88 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x74:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.665815:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360502400:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 142us (833us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.665941:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35939:x1875453360501888:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 18378us (18477us total) trans 4294970697 rc 0/0 00000100:00100000:3.0:1788572546.665950:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4540 00000100:00100000:1.0:1788572546.665953:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1079 00000020:00080000:3.0:1788572546.665954:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00010000:00010000:0.0:1788572546.666272:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.666280:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572546.666416:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970693 00000001:00080000:0.0:1788572546.666590:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970698 00000001:00080000:0.0:1788572546.666639:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970699 00000100:00100000:0.0:1788572546.666727:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5430380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382381568:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572546.666779:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572546.666817:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb0db240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.667156:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382381568 00000100:00100000:1.0:1788572546.667163:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382381568 00000100:00100000:1.0:1788572546.667216:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1788572546.667228:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382381568:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572546.667295:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382381568:12345-0@lo:1000:mdt00_000.0 Request processed in 71us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.667302:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:0.0:1788572546.667412:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967641 gen 2 00000004:00080000:0.0:1788572546.667418:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967635 committed transno 4294967641 00000004:00080000:0.0:1788572546.667448:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967636 committed transno 4294967641 00000004:00080000:0.0:1788572546.667463:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c84ec000, transno 4294967639 00000004:00080000:0.0:1788572546.667482:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c84eea00, transno 4294967640 00000004:00080000:0.0:1788572546.667485:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c84ef480, transno 4294967641 00000100:00100000:0.0:1788572546.667493:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5430380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382381568:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1788572546.667894:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970700 00000004:00080000:2.0:1788572546.668279:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:0.0:1788572546.668428:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5430380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382381696:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572546.668485:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572546.668515:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9b799c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572546.668565:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382381696 00000100:00100000:1.0:1788572546.668571:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382381696 00000100:00100000:1.0:1788572546.668626:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 341 00000100:00100000:1.0:1788572546.668638:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382381696:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572546.668829:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382381696:12345-0@lo:1000:mdt00_000.0 Request processed in 193us (342us total) trans 4294967643 rc 0/0 00000100:00100000:1.0:1788572546.668837:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 341 00000100:00100000:2.0:1788572546.669162:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5430380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382381696:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1788572546.669201:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669211:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669219:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669225:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.669230:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669239:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669254:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c3da6100) 00010000:00010000:2.0:1788572546.669257:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572546.669262:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669267:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb163e9d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669298:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669307:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669313:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669317:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.669320:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669325:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669333:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c29dab00) 00010000:00010000:2.0:1788572546.669335:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572546.669338:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669343:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb15fc24 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669364:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 3/0,1 mode: --/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669376:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669385:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669392:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669398:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669403:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669407:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.669410:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669415:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669422:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c29dab00) 00010000:00010000:2.0:1788572546.669425:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.669428:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669434:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163eab lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669455:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669461:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669468:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669472:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.669476:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669481:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669489:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c29dab00) 00010000:00010000:2.0:1788572546.669492:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572546.669494:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.669499:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163e96 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.669646:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35940:x1875453360502144:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 16160us (16360us total) trans 4294970701 rc 0/0 00000100:00100000:2.0:1788572546.669656:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4541 00000020:00080000:2.0:1788572546.669660:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1788572546.669827:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572546.669832:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572546.669878:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 160 count 101 00000040:00080000:1.0:1788572546.669884:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:1.0:1788572546.669996:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382381824:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572546.670072:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.675451:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.675525:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360502656 00000100:00100000:2.0:1788572546.675536:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360502656 00000100:00100000:2.0:1788572546.675598:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 954 00000100:00100000:2.0:1788572546.675608:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35939:x1875453360502656:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572546.675998:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x9b:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572546.676077:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970702 00010000:00010000:2.0:1788572546.676123:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.676139:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.676151:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.676160:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.676166:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.676172:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.676186:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1788572546.676275:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35939:x1875453360502656:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 668us (843us total) trans 4294970703 rc 0/0 00000100:00100000:2.0:1788572546.676283:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 954 00000020:00080000:2.0:1788572546.676288:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572546.676456:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572546.676463:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.676470:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572546.676474:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.676482:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.676496:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9b:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572546.676499:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572546.676504:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.676510:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163eb2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.676528:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572546.676530:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572546.676532:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572546.677144:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382381824:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572546.682779:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.682887:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360502784 00000100:00100000:2.0:1788572546.682897:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360502784 00000100:00100000:2.0:1788572546.683903:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4542 00000100:00100000:2.0:1788572546.683924:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35941:x1875453360502784:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572546.683960:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa8:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572546.684101:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35941:x1875453360502784:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 187us (1331us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.684108:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4542 00000100:00100000:3.0:1788572546.686692:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.686741:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.686776:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.686868:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360503168 00000100:00100000:1.0:1788572546.686878:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360503168 00000100:00100000:1.0:1788572546.686966:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 955 00000100:00100000:1.0:1788572546.686978:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35940:x1875453360503168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572546.687418:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x9a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572546.687472:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970704 00010000:00010000:1.0:1788572546.687518:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687536:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687549:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687557:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687563:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687569:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687589:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788572546.687622:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572546.687628:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687635:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572546.687639:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687647:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687663:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9a:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572546.687666:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572546.687671:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687677:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163eb9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.687692:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572546.687694:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572546.687696:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572546.687773:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35940:x1875453360503168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 798us (1029us total) trans 4294970705 rc 0/0 00000100:00100000:1.0:1788572546.687781:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 955 00000020:00080000:1.0:1788572546.687786:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788572546.688106:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360502912 00000100:00100000:0.0:1788572546.688112:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360502912 00000100:00100000:0.0:1788572546.688146:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360503296 00000100:00100000:0.0:1788572546.688149:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360503296 00000100:00100000:0.0:1788572546.688171:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4543 00000100:00100000:0.0:1788572546.688179:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35942:x1875453360502912:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572546.688212:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa3:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572546.688290:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35942:x1875453360502912:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 112us (1610us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.688297:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4543 00000100:00100000:0.0:1788572546.688312:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4544 00000100:00100000:0.0:1788572546.688318:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35943:x1875453360503296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572546.688331:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa5:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572546.688371:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35943:x1875453360503296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 53us (1595us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.688378:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4544 00000100:00100000:3.0:1788572546.693617:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.693704:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360503424 00000100:00100000:2.0:1788572546.693714:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360503424 00000100:00100000:2.0:1788572546.693778:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4545 00000100:00100000:2.0:1788572546.693789:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35941:x1875453360503424:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.693866:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693885:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693898:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693926:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693934:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693941:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693970:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693978:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.693984:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572546.694055:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa8:0x0]: returning size 3 00000004:00200000:2.0:1788572546.694060:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572546.694075:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694087:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694097:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694104:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694110:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.694114:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694121:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694132:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572546.694135:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.694140:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694147:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb163ec0 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694161:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694167:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694172:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694176:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.694179:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694184:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694201:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x5043330f (ffff9627e7a95a00) 00010000:00010000:2.0:1788572546.694204:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.694206:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694212:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ec7 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x5043330f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694226:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694232:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694237:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694242:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.694245:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694251:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694258:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:2.0:1788572546.694261:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572546.694264:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.694269:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163ece lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.694338:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35941:x1875453360503424:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 550us (731us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.694346:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4545 00000100:00100000:0.0:1788572546.698924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.699043:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360503552 00000100:00100000:3.0:1788572546.699055:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360503552 00000100:00100000:3.0:1788572546.699113:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4546 00000100:00100000:3.0:1788572546.699126:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35942:x1875453360503552:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.699218:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699237:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699251:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699269:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699276:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699284:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699318:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699325:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699331:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.699381:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa3:0x0]: returning size 3 00000004:00200000:3.0:1788572546.699386:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.699401:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699413:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699419:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699425:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699430:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.699435:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699473:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699485:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572546.699488:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.699494:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699501:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163ed5 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699514:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699524:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699529:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699534:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.699537:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699542:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699549:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x28a9d3bb (ffff9626c1d8f400) 00010000:00010000:3.0:1788572546.699552:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.699555:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699561:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163edc lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x28a9d3bb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699577:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699583:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699588:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699593:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.699597:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699602:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699609:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627ffc16700) 00010000:00010000:3.0:1788572546.699612:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.699615:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.699621:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163ee3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.699699:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35942:x1875453360503552:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 575us (795us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.699707:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4546 00000100:00100000:0.0:1788572546.704773:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.704888:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360504192 00000100:00100000:2.0:1788572546.704899:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360504192 00000100:00100000:2.0:1788572546.704973:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4547 00000100:00100000:2.0:1788572546.704983:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35942:x1875453360504192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.705039:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.705064:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163eea lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02e4 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705204:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705222:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705237:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572546.705250:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa3:0x0]: returning size 3 00010000:00010000:2.0:1788572546.705352:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705360:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705367:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705373:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572546.705377:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705385:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705397:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627ffc16700) 00010000:00010000:2.0:1788572546.705401:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572546.705406:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705413:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb163ef1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705467:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163eea lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02e4 expref: 82 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705479:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572546.705481:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572546.705485:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb163eea lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02e4 expref: 82 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.705495:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fc00, rc 0) 00000100:00100000:2.0:1788572546.705550:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35942:x1875453360504192:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 568us (788us total) trans 4294970706 rc 301/301 00000100:00100000:2.0:1788572546.705560:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4547 00000100:00100000:0.0:1788572546.709407:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.709463:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.709822:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360503936 00000100:00100000:3.0:1788572546.709833:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360503936 00000100:00100000:3.0:1788572546.709888:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360503808 00000100:00100000:3.0:1788572546.709892:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360503808 00000100:00100000:3.0:1788572546.709928:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4548 00000100:00100000:3.0:1788572546.709938:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35943:x1875453360503936:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.710051:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710070:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710083:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710119:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710126:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710132:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710160:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710168:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710174:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.710333:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4549 00000100:00100000:0.0:1788572546.710343:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35941:x1875453360503808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.710355:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.710368:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163f14 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02dd expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710460:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710470:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710478:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.710489:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa8:0x0]: returning size 3 00010000:00010000:0.0:1788572546.710590:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710598:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710605:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710611:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.710615:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710622:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710636:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:0.0:1788572546.710639:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572546.710645:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710651:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb163f1b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710698:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163f14 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02dd expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710709:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572546.710712:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572546.710716:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb163f14 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02dd expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.710725:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc 0) 00000100:00100000:0.0:1788572546.710787:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35941:x1875453360503808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 446us (1322us total) trans 4294970707 rc 301/301 00000100:00100000:0.0:1788572546.710794:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4549 00000004:00200000:3.0:1788572546.710955:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa5:0x0]: returning size 3 00000004:00200000:3.0:1788572546.710960:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.710970:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710981:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710987:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.710992:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711034:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.711039:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711045:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711055:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572546.711057:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.711060:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711065:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163eff lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711075:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711080:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711085:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711089:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.711091:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711172:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711187:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xac752f65 (ffff9626c1d8f400) 00010000:00010000:3.0:1788572546.711258:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.711263:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711269:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f06 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xac752f65 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711286:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711292:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711297:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711301:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.711304:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711308:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711316:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9627fb1e7800) 00010000:00010000:3.0:1788572546.711318:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.711320:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.711325:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f0d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.711372:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35943:x1875453360503936:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1435us (1973us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.711379:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4548 00000100:00100000:2.0:1788572546.720675:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.720728:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.720800:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360504320 00000100:00100000:3.0:1788572546.720812:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360504320 00000100:00100000:3.0:1788572546.720882:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360504576 00000100:00100000:3.0:1788572546.720886:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360504576 00000100:00100000:3.0:1788572546.720909:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4550 00000100:00100000:3.0:1788572546.720918:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35941:x1875453360504320:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572546.720967:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa8:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572546.721118:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35941:x1875453360504320:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 200us (455us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.721125:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4550 00000100:00100000:3.0:1788572546.721143:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4551 00000100:00100000:3.0:1788572546.721150:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35943:x1875453360504576:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.721160:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.721181:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163f29 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02eb expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721650:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721666:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721678:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.721690:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa5:0x0]: returning size 3 00010000:00010000:3.0:1788572546.721797:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721805:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721812:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721818:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.721823:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721831:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.721844:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9627fb1e7800) 00010000:00010000:3.0:1788572546.722607:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572546.722615:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.722624:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb163f30 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.722679:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163f29 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02eb expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.722690:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572546.722693:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572546.722697:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb163f29 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02eb expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.722706:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14c600, rc 0) 00000100:00100000:3.0:1788572546.722806:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35943:x1875453360504576:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1655us (2076us total) trans 4294970708 rc 301/301 00000100:00100000:3.0:1788572546.722813:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4551 00000100:00100000:0.0:1788572546.735816:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.735931:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.736501:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.736557:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360504704 00000100:00100000:3.0:1788572546.736568:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360504704 00000100:00100000:3.0:1788572546.736635:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360504832 00000100:00100000:3.0:1788572546.736639:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360504832 00000100:00100000:3.0:1788572546.736668:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4552 00000100:00100000:3.0:1788572546.736678:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35942:x1875453360504704:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572546.736718:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa3:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572546.736964:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35942:x1875453360504704:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 167us (1043us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.736975:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4552 00000100:00100000:3.0:1788572546.736997:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4553 00000100:00100000:3.0:1788572546.737032:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35941:x1875453360504832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.737045:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.737066:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f3e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02f2 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737391:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737411:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737427:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737462:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737470:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737477:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737825:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163f53 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737836:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163f53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737843:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163f53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.737882:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572546.737886:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.737908:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163f53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737929:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737935:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737947:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737953:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.737958:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737967:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737981:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572546.737984:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.737989:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.737996:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb163f45 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738055:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738061:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738067:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738071:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.738073:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738082:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738090:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8f400) 00010000:00010000:3.0:1788572546.738092:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.738095:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738113:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb163f4c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738152:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb163f3e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02f2 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738164:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163f53 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02f2 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.738171:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.738173:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572546.738176:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d200, rc 0) 00000100:00100000:3.0:1788572546.741173:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35941:x1875453360504832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4155us (5227us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.741186:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4553 00000100:00100000:3.0:1788572546.742469:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360504960 00000100:00100000:3.0:1788572546.742483:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360504960 00000100:00100000:3.0:1788572546.742545:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4554 00000100:00100000:3.0:1788572546.742555:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35943:x1875453360504960:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572546.742594:0:5839:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa5:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572546.742708:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35943:x1875453360504960:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 154us (6208us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.742716:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4554 00000100:00100000:3.0:1788572546.755627:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.755665:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.755699:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360505344 00000100:00100000:3.0:1788572546.755708:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360505344 00000100:00100000:3.0:1788572546.755758:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360505216 00000100:00100000:3.0:1788572546.755761:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360505216 00000100:00100000:3.0:1788572546.755786:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4555 00000100:00100000:3.0:1788572546.755795:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35942:x1875453360505344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.755815:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.755837:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163f5a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0300 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.755930:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.755946:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.755957:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.755973:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.755980:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.755986:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756610:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163f6f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756623:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163f6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756632:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163f6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.756660:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572546.756664:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.756683:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163f6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756694:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756701:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756707:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756712:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.756716:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756723:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756736:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572546.756739:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.756744:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756750:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1600/0x85ace67feb163f61 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756761:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756767:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756773:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756777:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ae1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.756780:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756785:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756791:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8f200) 00010000:00010000:3.0:1788572546.756794:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.756796:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756802:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1200/0x85ace67feb163f68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756838:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb163f5a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0300 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756854:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163f6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0300 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.756861:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.756863:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572546.756865:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ae1800, rc 0) 00000100:00100000:3.0:1788572546.756938:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35942:x1875453360505344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1144us (1322us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.756950:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4555 00000100:00100000:3.0:1788572546.756975:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4556 00000100:00100000:3.0:1788572546.756984:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35941:x1875453360505216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.756994:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.756996:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572546.757033:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb163f53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02f2 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.757045:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.757048:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572546.757056:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163f76 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f02f9 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572546.757768:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x55:0x0], index 1: 0 00010000:00010000:3.0:1788572546.760771:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163f7d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.760795:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163f7d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.760806:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163f7d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.760895:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163f76 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02f9 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.760924:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163f7d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02f9 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.760934:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f600) 00010000:00010000:3.0:1788572546.760937:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572546.760941:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572546.761036:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35941:x1875453360505216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4053us (5369us total) trans 4294970709 rc 0/0 00000100:00100000:3.0:1788572546.761047:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4556 00000100:00100000:0.0:1788572546.764962:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.765107:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360505600 00000100:00100000:1.0:1788572546.765118:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360505600 00000100:00100000:1.0:1788572546.765200:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4557 00000100:00100000:1.0:1788572546.765214:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35941:x1875453360505600:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.765275:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572546.765302:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb163f7d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f02f9 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.765319:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f600) 00010000:00010000:1.0:1788572546.765334:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ae1800/0x85ace67feb163f6f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0300 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.765348:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572546.765351:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572546.765373:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.765386:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 175 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.765395:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 4/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.765431:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 175 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.765455:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 175 pid: 5842 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.765469:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382381952:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572546.765568:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.769519:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.769594:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360505728 00000100:00100000:3.0:1788572546.769604:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360505728 00010000:00010000:3.0:1788572546.769647:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 176 pid: 5842 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.769693:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1080 00000100:00100000:3.0:1788572546.769703:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:2349:x1875453360505728:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572546.769719:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572546.769727:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 176 pid: 5842 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.769739:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 176 pid: 5842 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.769761:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.769766:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.769776:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.769784:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.769804:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572546.769854:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:2349:x1875453360505728:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 152us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.769862:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1080 00010000:00010000:1.0:1788572546.770850:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb15dc0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ede63 expref: 175 pid: 5842 timeout: 994 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.770869:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382381952:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.770905:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.770926:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.770933:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.770959:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.770969:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.770978:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771248:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771256:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771263:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771268:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.771272:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771280:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771291:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572546.771293:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.771298:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771303:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb163f8b lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771312:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771317:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771322:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771327:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.771329:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771334:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771340:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xd7559a80 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572546.771342:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572546.771345:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.771350:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163f92 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0xd7559a80 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.771420:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35941:x1875453360505600:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 6209us (6468us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572546.771429:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4557 00000100:00100000:1.0:1788572546.775981:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.776047:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.776169:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360505856 00000100:00100000:3.0:1788572546.776180:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360505856 00000100:00100000:3.0:1788572546.776247:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360506112 00000100:00100000:3.0:1788572546.776251:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360506112 00000100:00100000:3.0:1788572546.776278:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4558 00000100:00100000:3.0:1788572546.776287:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35941:x1875453360505856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.776308:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.776330:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163f99 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0307 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.776373:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.776386:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.776397:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.776511:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163f99 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.776528:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.776536:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:3.0:1788572546.776540:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572546.776543:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572546.776595:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35941:x1875453360505856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 308us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.776602:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4558 00000100:00100000:3.0:1788572546.776620:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4559 00000100:00100000:3.0:1788572546.776628:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35942:x1875453360506112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.776636:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.776645:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163fa7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f030e expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572546.777065:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x56:0x0], index 0: 0 00010000:00010000:3.0:1788572546.777259:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163fae lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.777269:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163fae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.777277:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163fae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.777328:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163fa7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f030e expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.777341:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163fae lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f030e expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.777349:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f400) 00010000:00010000:3.0:1788572546.777351:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572546.777354:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572546.777395:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35942:x1875453360506112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 767us (1347us total) trans 4294970710 rc 0/0 00000100:00100000:3.0:1788572546.777402:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4559 00000100:00100000:3.0:1788572546.783963:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.786347:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360506240 00000100:00100000:1.0:1788572546.786364:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360506240 00000100:00100000:0.0:1788572546.786746:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.787748:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360506496 00000100:00100000:1.0:1788572546.787760:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360506496 00000100:00100000:1.0:1788572546.787837:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4560 00000100:00100000:1.0:1788572546.787860:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35941:x1875453360506496:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.787976:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.787994:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.788106:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.789576:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.789649:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360506752 00000100:00100000:3.0:1788572546.789657:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360506752 00000100:00100000:3.0:1788572546.789716:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4561 00000100:00100000:3.0:1788572546.789727:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35943:x1875453360506752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.789748:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.789769:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163fc3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f031c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572546.790517:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x56:0x0], index 1: 0 00010000:00010000:3.0:1788572546.790734:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163fca lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.790749:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163fca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.790760:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163fca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.790823:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb163fc3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f031c expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.790843:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163fca lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f031c expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.790944:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572546.790947:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572546.790951:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc400, rc 0) 00000100:00100000:3.0:1788572546.791045:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35943:x1875453360506752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1320us (1475us total) trans 4294970711 rc 0/0 00000100:00100000:3.0:1788572546.791053:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4561 00000100:00100000:2.0:1788572546.794142:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4562 00000100:00100000:2.0:1788572546.794159:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35942:x1875453360506240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572546.794181:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572546.794457:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163fd8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0315 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.794533:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.794548:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.794560:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.794573:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.794580:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.794586:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572546.795615:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].71, 88 off14032 00000004:00010000:0.0:1788572546.795715:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000097369413 reply state 000000007dff8613 transno 4294970712 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.795726:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572546.795864:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35941:x1875453360506496:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 8015us (9120us total) trans 4294970712 rc 0/0 00000100:00100000:0.0:1788572546.795873:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4560 00010000:00010000:0.0:1788572546.795989:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163fed lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796032:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163fed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796042:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163fed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.796063:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:0.0:1788572546.796067:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.796082:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163fed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796093:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796111:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796118:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796124:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.796128:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796135:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796146:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572546.796149:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.796154:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796164:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb163fdf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796177:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796183:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796200:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796206:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.796209:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796215:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796222:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9627e7a95a00) 00010000:00010000:0.0:1788572546.796225:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572546.796228:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796235:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb163fe6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796267:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb163fd8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0315 expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796285:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163fed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0315 expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.796292:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572546.796294:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572546.796297:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc600, rc 0) 00000100:00100000:0.0:1788572546.796323:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35942:x1875453360506240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2171us (12370us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.796330:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4562 00010000:00010000:0.0:1788572546.800372:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.800391:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.800400:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.800406:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.800410:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.800418:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.800447:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:0.0:1788572546.800451:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572546.800456:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.800463:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb163fbc lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.803154:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.803199:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.803223:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.803256:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360507008 00000100:00100000:3.0:1788572546.803265:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360507008 00000100:00100000:3.0:1788572546.803311:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360507136 00000100:00100000:3.0:1788572546.803314:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360507136 00000100:00100000:3.0:1788572546.803330:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360507264 00000100:00100000:3.0:1788572546.803332:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360507264 00000100:00100000:3.0:1788572546.803349:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4563 00000100:00100000:3.0:1788572546.803359:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35942:x1875453360507008:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.803422:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572546.803460:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb163fae lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f030e expref: 93 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803480:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f400) 00010000:00010000:3.0:1788572546.803500:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb163fed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0315 expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803509:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.803513:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572546.803537:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803551:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803560:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803574:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803585:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803591:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803827:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803836:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803842:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803848:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.803852:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803860:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803869:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.803872:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.803875:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803881:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb163ff4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803890:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803896:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803902:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803907:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.803909:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803915:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803921:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x86d5d145 (ffff9626c1d8f700) 00010000:00010000:3.0:1788572546.803924:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.803926:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.803931:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb163ffb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x86d5d145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.804041:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35942:x1875453360507008:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 682us (895us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572546.804049:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4563 00000100:00100000:3.0:1788572546.804071:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4564 00000100:00100000:3.0:1788572546.804078:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35941:x1875453360507136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.804089:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.804100:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0323 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804142:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804154:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804174:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:3.0:1788572546.804176:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.804179:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572546.804202:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35941:x1875453360507136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 125us (1013us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.804208:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4564 00000100:00100000:3.0:1788572546.804224:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4565 00000100:00100000:3.0:1788572546.804231:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35943:x1875453360507264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.804238:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.804248:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164009 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f032a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804287:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804295:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804302:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804317:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804323:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804328:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804551:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb16401e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804561:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb16401e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804567:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb16401e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572546.804588:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572546.804591:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572546.804608:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb16401e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804618:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804623:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804629:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804638:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.804641:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804647:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804656:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572546.804658:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.804661:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804667:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164010 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804676:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804682:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804687:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804691:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.804694:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804700:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804711:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8fe00) 00010000:00010000:3.0:1788572546.804714:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.804716:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804722:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb164017 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804750:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164009 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f032a expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804763:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb16401e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f032a expref: 91 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.804770:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.804772:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572546.804774:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d800, rc 0) 00000100:00100000:3.0:1788572546.804800:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35943:x1875453360507264:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 570us (1575us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.804806:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4565 00000100:00100000:0.0:1788572546.808611:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.808683:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360507392 00000100:00100000:3.0:1788572546.808694:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360507392 00000100:00100000:3.0:1788572546.808760:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1272 00000100:00100000:3.0:1788572546.808770:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360507392:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.809146:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360507392:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 376us (544us total) trans 4294967644 rc 0/0 00000100:00100000:3.0:1788572546.809157:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1272 00000100:00100000:3.0:1788572546.811907:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.813872:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.813951:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360507520 00000100:00100000:3.0:1788572546.813961:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360507520 00000100:00100000:3.0:1788572546.814052:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4566 00000100:00100000:3.0:1788572546.814063:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35942:x1875453360507520:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.814084:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.814108:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164025 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0331 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.814145:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.814159:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.814170:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.814285:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164025 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 93 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.814307:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 92 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.814315:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572546.814318:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572546.814322:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14c600, rc 0) 00000100:00100000:3.0:1788572546.814380:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35942:x1875453360507520:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 320us (516us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.814388:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4566 00000100:00100000:3.0:1788572546.817830:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.817930:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360508288 00000100:00100000:1.0:1788572546.817940:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360508288 00000100:00100000:1.0:1788572546.818047:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4567 00000100:00100000:1.0:1788572546.818060:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35942:x1875453360508288:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.818153:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.818170:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.818181:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572546.818627:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].74, 88 off14224 00000004:00010000:1.0:1788572546.818720:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000839542f reply state 000000002e344c18 transno 4294970713 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.818729:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.818833:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35942:x1875453360508288:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 776us (1012us total) trans 4294970713 rc 0/0 00000100:00100000:1.0:1788572546.818841:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4567 00000100:00100000:2.0:1788572546.822519:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.822725:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360507776 00000100:00100000:0.0:1788572546.822736:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360507776 00000100:00100000:3.0:1788572546.822777:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360508032 00000100:00100000:3.0:1788572546.822782:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360508032 00000100:00100000:0.0:1788572546.822799:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 956 00000100:00100000:0.0:1788572546.822809:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35941:x1875453360507776:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.822810:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4568 00000100:00100000:3.0:1788572546.822816:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35943:x1875453360508032:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.822874:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788572546.822901:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb163fca lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f031c expref: 95 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.822930:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572546.822947:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb16401e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f032a expref: 94 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.822955:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.822959:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572546.822979:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.822991:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823000:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823029:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823036:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823042:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823212:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823220:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823226:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823232:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.823239:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.823244:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35941:x1875453360507776:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 436us (11349us total) trans 4294970714 rc 0/0 00010000:00010000:3.0:1788572546.823247:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.823252:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 956 00010000:00010000:3.0:1788572546.823256:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572546.823258:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.823262:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823268:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb16403a lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823277:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823284:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823290:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823294:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572546.823297:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823303:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823310:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x615a1c1 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572546.823312:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572546.823315:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.823320:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb164041 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x615a1c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572546.823468:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35943:x1875453360508032:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 649us (958us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572546.823477:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4568 00010000:00010000:1.0:1788572546.823719:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.823727:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.823733:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.823739:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572546.823742:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.823748:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.823757:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:1.0:1788572546.823760:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572546.823763:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.823768:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164033 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.824064:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.824491:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360508416 00000100:00100000:3.0:1788572546.824499:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360508416 00000100:00100000:3.0:1788572546.824540:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4569 00000100:00100000:3.0:1788572546.824548:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35942:x1875453360508416:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.824562:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.824574:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0338 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.824618:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 93 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.824631:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 93 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.824649:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572546.824652:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.824654:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572546.824697:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35942:x1875453360508416:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 149us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.824704:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4569 00000100:00100000:0.0:1788572546.829290:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.829333:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.829483:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360508544 00000100:00100000:3.0:1788572546.829493:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360508544 00000100:00100000:3.0:1788572546.829545:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1273 00000100:00100000:3.0:1788572546.829556:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360508544:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.829883:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360508544:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 327us (601us total) trans 4294967646 rc 0/0 00000100:00100000:3.0:1788572546.829891:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1273 00000100:00100000:1.0:1788572546.830451:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360508672 00000100:00100000:1.0:1788572546.830457:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360508672 00000100:00100000:1.0:1788572546.830502:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4570 00000100:00100000:1.0:1788572546.830513:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35943:x1875453360508672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.830528:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.830548:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16404f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f033f expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.830584:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.830598:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.830609:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.830704:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16404f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 95 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.830720:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.830729:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572546.830732:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572546.830735:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572546.830774:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35943:x1875453360508672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 265us (1440us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.830784:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4570 00000100:00100000:0.0:1788572546.834905:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.835046:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360508928 00000100:00100000:3.0:1788572546.835056:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360508928 00000100:00100000:3.0:1788572546.835113:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4571 00000100:00100000:3.0:1788572546.835123:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35941:x1875453360508928:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.835251:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35941:x1875453360508928:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 129us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.835259:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4571 00000100:00100000:3.0:1788572546.839761:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.839814:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.839923:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360509440 00000100:00100000:2.0:1788572546.839935:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360509440 00000100:00100000:2.0:1788572546.839994:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 957 00000100:00100000:2.0:1788572546.840042:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35942:x1875453360509440:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.840518:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35942:x1875453360509440:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 513us (701us total) trans 4294970715 rc 0/0 00000100:00100000:2.0:1788572546.840527:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 957 00000100:00100000:0.0:1788572546.840725:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360509568 00000100:00100000:0.0:1788572546.840731:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360509568 00000100:00100000:0.0:1788572546.840776:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4572 00000100:00100000:0.0:1788572546.840784:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35941:x1875453360509568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.840796:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.840817:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb16405d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0346 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.840878:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb164064 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.840896:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb164064 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.840907:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb164064 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.840932:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.840946:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb164064 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.840970:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb16405d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0346 expref: 97 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.840988:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb164064 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0346 expref: 96 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.840997:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:0.0:1788572546.841000:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572546.841045:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd600, rc 0) 00000100:00100000:0.0:1788572546.841070:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35941:x1875453360509568:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 287us (1319us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.841077:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4572 00000100:00100000:2.0:1788572546.843240:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.843414:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360509184 00000100:00100000:1.0:1788572546.843422:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360509184 00000100:00100000:1.0:1788572546.843501:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4573 00000100:00100000:1.0:1788572546.843513:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35943:x1875453360509184:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.843608:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.843623:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.843634:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572546.844160:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].72, 88 off14120 00000004:00010000:1.0:1788572546.844326:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009484ab14 reply state 00000000ae0f68bf transno 4294970716 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.844336:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572546.844492:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35943:x1875453360509184:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 980us (1257us total) trans 4294970716 rc 0/0 00000100:00100000:1.0:1788572546.844500:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4573 00010000:00010000:0.0:1788572546.848791:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.848808:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.848815:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.848822:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572546.848826:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.848835:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.848853:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:0.0:1788572546.848856:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572546.848862:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.848868:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16406b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.849720:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.849811:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.850222:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360509824 00000100:00100000:1.0:1788572546.850232:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360509824 00000100:00100000:3.0:1788572546.850356:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360509696 00000100:00100000:3.0:1788572546.850361:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360509696 00000100:00100000:3.0:1788572546.850410:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4574 00000100:00100000:3.0:1788572546.850420:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35943:x1875453360509696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572546.850458:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572546.850483:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f034d expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.850551:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 98 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.850566:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 98 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572546.850587:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:3.0:1788572546.850590:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572546.850593:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572546.850653:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35943:x1875453360509696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 234us (939us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.850660:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4574 00000100:00100000:1.0:1788572546.851175:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1274 00000100:00100000:1.0:1788572546.851204:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360509824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.851214:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.851229:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0354 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572546.851253:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x55:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7382, blocks: 0x1 00010000:00010000:1.0:1788572546.851269:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=894s v=(0 0) 00010000:00010000:1.0:1788572546.851274:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.851283:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.851296:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.851307:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572546.851336:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360509824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 152us (1524us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.851342:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1274 00000100:00100000:2.0:1788572546.854768:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.854849:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360510080 00000100:00100000:0.0:1788572546.854858:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360510080 00000100:00100000:0.0:1788572546.854927:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4575 00000100:00100000:0.0:1788572546.854937:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35942:x1875453360510080:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.855090:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35942:x1875453360510080:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 153us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.855098:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4575 00000100:00100000:3.0:1788572546.859054:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.859181:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360510336 00000100:00100000:1.0:1788572546.859202:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360510336 00000100:00100000:1.0:1788572546.859275:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4576 00000100:00100000:1.0:1788572546.859288:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35941:x1875453360510336:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.859402:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35941:x1875453360510336:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 117us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.859410:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4576 00000100:00100000:3.0:1788572546.862448:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.862533:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360510464 00000100:00100000:0.0:1788572546.862546:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360510464 00000100:00100000:0.0:1788572546.862610:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1275 00000100:00100000:0.0:1788572546.862619:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360510464:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.862967:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360510464:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 348us (537us total) trans 4294967645 rc 0/0 00000100:00100000:0.0:1788572546.862977:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1275 00000100:00100000:3.0:1788572546.865490:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.865955:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360510592 00000100:00100000:0.0:1788572546.865966:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360510592 00000100:00100000:0.0:1788572546.866067:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4577 00000100:00100000:0.0:1788572546.866077:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35942:x1875453360510592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.866098:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.866120:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164080 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f035b expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.866211:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164087 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.866226:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164087 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.866237:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164087 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.866275:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.866296:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164087 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.866325:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164080 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f035b expref: 100 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.866343:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164087 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f035b expref: 99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.866351:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:0.0:1788572546.866355:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572546.866359:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc600, rc 0) 00000100:00100000:0.0:1788572546.866425:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35942:x1875453360510592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 349us (946us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.866432:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4577 00000100:00100000:3.0:1788572546.868797:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.868878:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360510720 00000100:00100000:0.0:1788572546.868888:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360510720 00000100:00100000:0.0:1788572546.868951:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1276 00000100:00100000:0.0:1788572546.868961:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35941:x1875453360510720:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.868978:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.868995:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0362 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.869056:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.869066:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.869071:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.869076:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.869111:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.869125:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 14 pid: 6681 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.869145:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382382080:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572546.869252:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb0db240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.874425:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.874468:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.874535:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.874618:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360511104 00000100:00100000:0.0:1788572546.874627:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360511104 00000100:00100000:0.0:1788572546.874687:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1277 00000100:00100000:0.0:1788572546.874698:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360511104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.874717:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.874739:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0369 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572546.874767:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x56:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7382, blocks: 0x1 00010000:00010000:0.0:1788572546.874786:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=894s v=(0 0) 00010000:00010000:0.0:1788572546.874791:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.874799:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.874816:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.874828:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bca00, rc 0) 00000100:00100000:0.0:1788572546.874892:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360511104:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 195us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.874899:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1277 00000100:00100000:2.0:1788572546.875144:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360510976 00000100:00100000:2.0:1788572546.875149:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360510976 00000100:00100000:2.0:1788572546.875182:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 958 00000100:00100000:2.0:1788572546.875202:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35943:x1875453360510976:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.875575:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35943:x1875453360510976:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 385us (1158us total) trans 4294970717 rc 0/0 00000100:00100000:2.0:1788572546.875583:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 958 00000100:00100000:1.0:1788572546.875724:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360511232 00000100:00100000:1.0:1788572546.875730:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360511232 00010000:00010000:1.0:1788572546.875769:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 15 pid: 6681 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.875813:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1081 00000100:00100000:1.0:1788572546.875823:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360511232:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.875834:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.875861:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 15 pid: 6681 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.875875:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 15 pid: 6681 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.875895:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.875899:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.875907:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.875928:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.875964:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.875979:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382382208:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572546.876065:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572546.876100:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb164079 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0354 expref: 15 pid: 6681 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.876115:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382382080:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.876139:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.876149:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bce00, rc 0) 00000100:00100000:1.0:1788572546.876171:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35941:x1875453360510720:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 7212us (7384us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.876178:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1276 00000100:00100000:3.0:1788572546.880378:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.880594:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360511488 00000100:00100000:0.0:1788572546.880605:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360511488 00000100:00100000:0.0:1788572546.880670:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4578 00000100:00100000:0.0:1788572546.880681:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35943:x1875453360511488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572546.880811:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35943:x1875453360511488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 131us (444us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.880819:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4578 00000100:00100000:1.0:1788572546.880959:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382382208:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572546.880996:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.881057:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360511232:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5236us (6521us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.881065:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1081 00000100:00100000:3.0:1788572546.883502:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.883588:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360511744 00000100:00100000:1.0:1788572546.883596:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360511744 00000100:00100000:1.0:1788572546.883669:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4579 00000100:00100000:1.0:1788572546.883683:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35942:x1875453360511744:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.883795:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35942:x1875453360511744:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 118us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.883803:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4579 00000100:00100000:2.0:1788572546.886948:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.887062:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360511872 00000100:00100000:0.0:1788572546.887073:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360511872 00000100:00100000:0.0:1788572546.887131:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4580 00000100:00100000:0.0:1788572546.887141:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35943:x1875453360511872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.887161:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.887183:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16409c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0370 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.887268:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1640a3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.887283:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1640a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.887295:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1640a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572546.887335:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572546.887357:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1640a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.887387:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16409c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0370 expref: 102 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.887411:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1640a3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0370 expref: 101 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.887419:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:0.0:1788572546.887422:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572546.887426:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bde00, rc 0) 00000100:00100000:0.0:1788572546.887508:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35943:x1875453360511872:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 368us (572us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.887515:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4580 00000100:00100000:3.0:1788572546.892820:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.892895:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.893043:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512128 00000100:00100000:1.0:1788572546.893053:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512128 00002000:00100000:1.0:1788572546.893122:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x55:0x0] (0->2) req@ffff9626c4bb9180 x1875453360512128/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.893134:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bb9180 with x1875453360512128 ext(0->2) 00002000:00010000:1.0:1788572546.893139:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bb9180 00000100:00100000:1.0:1788572546.893177:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 285 00000100:00100000:1.0:1788572546.893198:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360512128:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.893670:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360512128:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 484us (773us total) trans 4294967646 rc 0/0 00000100:00100000:1.0:1788572546.893679:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 285 00002000:00100000:1.0:1788572546.893686:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x55:0x0] (0->2) req@ffff9626c4bb9180 x1875453360512128/t4294967646(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.893695:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bb9180 with x1875453360512128 ext(0->2) 00002000:00010000:1.0:1788572546.893698:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bb9180 00000100:00100000:0.0:1788572546.893872:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512000 00000100:00100000:0.0:1788572546.893878:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512000 00000100:00100000:0.0:1788572546.893909:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1278 00000100:00100000:0.0:1788572546.893917:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35942:x1875453360512000:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572546.893927:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572546.893944:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0377 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.893972:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.893981:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.893985:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572546.893990:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 4/0,0 mode: --/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.894049:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.894063:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572546.894077:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382382336:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572546.894133:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb0db240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.900666:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.900753:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512256 00000100:00100000:1.0:1788572546.900764:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512256 00000100:00100000:1.0:1788572546.900836:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1279 00000100:00100000:1.0:1788572546.900849:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360512256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.900866:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.900887:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f037e expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572546.900922:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x56:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7382, blocks: 0x1 00010000:00010000:1.0:1788572546.900942:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=894s v=(0 0) 00010000:00010000:1.0:1788572546.900947:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.900956:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 16 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.900973:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 16 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.900986:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:0.0:1788572546.901383:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360512256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 536us (726us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.901391:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1279 00000100:00100000:3.0:1788572546.904333:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.904366:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.904457:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512384 00000100:00100000:0.0:1788572546.904467:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512384 00000100:00100000:0.0:1788572546.904525:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512640 00000100:00100000:0.0:1788572546.904529:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512640 00000100:00100000:0.0:1788572546.904560:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4581 00000100:00100000:0.0:1788572546.904570:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35941:x1875453360512384:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.907207:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4582 00000100:00100000:2.0:1788572546.907224:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35943:x1875453360512640:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572546.907347:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35943:x1875453360512640:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 127us (2979us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572546.907355:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4582 00000100:00100000:3.0:1788572546.915588:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.915661:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.916183:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512768 00000100:00100000:1.0:1788572546.916205:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512768 00000100:00100000:1.0:1788572546.916281:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1280 00000100:00100000:1.0:1788572546.916294:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35943:x1875453360512768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.916313:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572546.916337:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0385 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.916369:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.916378:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.916382:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.916387:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.916421:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.916432:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.916458:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382382464:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572546.916533:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572546.920558:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382382336:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572546.920598:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 3/0,0 mode: --/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572546.920619:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bda00, rc 0) 00000100:00100000:0.0:1788572546.920682:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35942:x1875453360512000:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 26766us (27871us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.920696:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1278 00000100:00100000:2.0:1788572546.921139:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360512896 00000100:00100000:2.0:1788572546.921148:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360512896 00010000:00010000:2.0:1788572546.921183:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.921230:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1082 00000100:00100000:2.0:1788572546.921239:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360512896:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572546.921252:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572546.921273:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.921285:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.921308:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164095 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0369 expref: 14 pid: 10416 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.921322:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.921326:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572546.921335:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.921345:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572546.921379:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572546.921399:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f90380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382382592:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572546.921468:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff5483c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.924984:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.925114:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382382464:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572546.925168:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.925204:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97400, rc 0) 00000100:00100000:1.0:1788572546.925269:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35943:x1875453360512768:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 8979us (9690us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.925278:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1280 00000100:00100000:1.0:1788572546.927975:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f90380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382382592:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572546.928036:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572546.928089:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360512896:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6850us (12426us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.928100:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1082 00000100:00100000:1.0:1788572546.928131:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513024 00000100:00100000:1.0:1788572546.928137:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513024 00010000:00010000:1.0:1788572546.928200:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.928236:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1083 00000100:00100000:1.0:1788572546.928248:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360513024:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572546.928262:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572546.928290:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.928303:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.928327:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000093fd62da/0x85ace67feb1640b1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f037e expref: 17 pid: 6679 timeout: 994 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.928344:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.928348:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572546.928357:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.928367:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572546.928409:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572546.928425:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382382720:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572546.928493:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572546.940316:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.940415:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572546.940715:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382382720:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572546.940753:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572546.940816:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360513024:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 12571us (15848us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572546.940827:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1083 00000100:00100000:1.0:1788572546.945048:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513152 00000100:00100000:1.0:1788572546.945068:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513152 00002000:00100000:1.0:1788572546.945147:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x56:0x0] (0->2) req@ffff9626c4bb9880 x1875453360513152/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.945168:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bb9880 with x1875453360513152 ext(0->2) 00002000:00010000:1.0:1788572546.945174:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4bb9880 00000100:00100000:1.0:1788572546.945217:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513280 00000100:00100000:1.0:1788572546.945222:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513280 00002000:00100000:1.0:1788572546.945243:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x56:0x0] (0->2) req@ffff9626c4bbb480 x1875453360513280/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/0 e 0 to 0 dl 1788572557 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.945251:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bbb480 with x1875453360513280 ext(0->2) 00002000:00010000:1.0:1788572546.945254:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bbb480 00000100:00100000:1.0:1788572546.945324:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 286 00000100:00100000:1.0:1788572546.945337:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360513152:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.945870:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2347:x1875453360513152:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 536us (5561us total) trans 4294967647 rc 0/0 00000100:00100000:1.0:1788572546.945879:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 286 00002000:00100000:1.0:1788572546.945886:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x56:0x0] (0->2) req@ffff9626c4bb9880 x1875453360513152/t4294967647(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.945893:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bb9880 with x1875453360513152 ext(0->2) 00002000:00010000:1.0:1788572546.945897:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4bb9880 00000100:00100000:1.0:1788572546.945926:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 287 00000100:00100000:1.0:1788572546.945938:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360513280:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.946189:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2349:x1875453360513280:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 255us (5773us total) trans 4294967647 rc 0/0 00000100:00100000:1.0:1788572546.946196:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 287 00002000:00100000:1.0:1788572546.946203:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x56:0x0] (0->2) req@ffff9626c4bbb480 x1875453360513280/t4294967647(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:207/0 lens 496/448 e 0 to 0 dl 1788572557 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572546.946211:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bbb480 with x1875453360513280 ext(0->2) 00002000:00010000:1.0:1788572546.946214:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bbb480 00000100:00100000:2.0:1788572546.950779:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572546.950827:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.950908:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513408 00000100:00100000:1.0:1788572546.950920:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513408 00000100:00100000:1.0:1788572546.951034:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513536 00000100:00100000:1.0:1788572546.951037:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513536 00000100:00100000:1.0:1788572546.951064:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4583 00000100:00100000:1.0:1788572546.951075:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35942:x1875453360513408:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.951223:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35942:x1875453360513408:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 151us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.951230:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4583 00000100:00100000:1.0:1788572546.951337:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4584 00000100:00100000:1.0:1788572546.951348:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35943:x1875453360513536:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572546.951405:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35943:x1875453360513536:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 61us (577us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572546.951412:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4584 00000100:00100000:1.0:1788572546.955136:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.955182:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572546.955274:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513664 00000100:00100000:3.0:1788572546.955285:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513664 00000100:00100000:3.0:1788572546.955337:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360513792 00000100:00100000:3.0:1788572546.955340:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360513792 00000100:00100000:3.0:1788572546.955365:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1281 00000100:00100000:3.0:1788572546.955373:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360513664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572546.955451:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1282 00000100:00100000:3.0:1788572546.955460:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360513792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1788572546.994703:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967645 is committed 00000001:00080000:2.0:1788572546.994723:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967647 is committed 00000001:00080000:2.0:1788572546.994754:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967643 is committed 00000100:00100000:3.0:1788572546.996268:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360513792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 40798us (41074us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572546.996282:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1282 00000100:00100000:1.0:1788572546.999675:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572546.999752:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514048 00000100:00100000:1.0:1788572546.999762:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514048 00000100:00100000:1.0:1788572546.999814:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4585 00000100:00100000:1.0:1788572546.999824:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35943:x1875453360514048:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572546.999899:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.000267:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1640a3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0370 expref: 102 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.000286:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.000291:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572547.000326:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.000341:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.000351:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572547.000560:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008dcbd233 reply state 00000000635b20e6 transno 4294970718 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.000570:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572547.000661:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35943:x1875453360514048:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 838us (995us total) trans 4294970718 rc 0/0 00000100:00100000:1.0:1788572547.000669:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4585 00010000:00010000:1.0:1788572547.005979:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.005999:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.006032:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.006043:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.006048:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.006057:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.006077:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.006080:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.006087:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.006092:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1640bf lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.009835:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.011827:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514176 00000100:00100000:0.0:1788572547.011841:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514176 00000100:00100000:0.0:1788572547.011900:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1283 00000100:00100000:0.0:1788572547.011921:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360514176:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572547.012214:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x56:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572547.012218:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x56:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:0.0:1788572547.012220:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x56:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:0.0:1788572547.012222:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x56:0x0] updating lvb ctime from disk: 1788572546 -> 1788572526 00002000:00010000:0.0:1788572547.012225:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x56:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572547.012288:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360514176:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 379us (2464us total) trans 4294967648 rc 0/0 00000100:00100000:0.0:1788572547.012297:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1283 00000100:00100000:1.0:1788572547.017450:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.017494:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514432 00000100:00100000:1.0:1788572547.017504:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514432 00000100:00100000:1.0:1788572547.017569:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4586 00000100:00100000:1.0:1788572547.017580:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35943:x1875453360514432:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.017646:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.017666:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 102 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.017676:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.017718:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 102 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.017733:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 102 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.017748:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382382848:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.017832:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb9c0 going to sleep for 11 seconds 00000100:00080000:1.0:1788572547.017848:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572547.022704:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382382848:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572547.022738:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.023937:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.024508:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514560 00000100:00100000:2.0:1788572547.024517:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514560 00010000:00010000:2.0:1788572547.024563:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 103 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.024600:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1084 00000100:00100000:2.0:1788572547.024609:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:2349:x1875453360514560:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572547.024624:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.024631:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 103 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.024644:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 103 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.024673:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164056 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f033f expref: 103 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.024691:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:2.0:1788572547.024697:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.024707:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.024715:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.024740:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572547.024780:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:2349:x1875453360514560:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 173us (849us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.024785:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1084 00010000:00010000:2.0:1788572547.025510:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025522:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025552:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025559:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025566:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025572:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.025577:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025585:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025596:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:2.0:1788572547.025599:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572547.025603:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.025609:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1640c6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.025664:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35943:x1875453360514432:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8085us (8232us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572547.025673:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4586 00000100:00100000:1.0:1788572547.030030:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.030065:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514688 00000100:00100000:1.0:1788572547.030074:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514688 00000100:00100000:1.0:1788572547.030125:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1284 00000100:00100000:1.0:1788572547.030135:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360514688:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.030180:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/1,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.030214:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.030223:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x56:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788572547.030228:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.030233:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 4/1,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.030275:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.030288:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 12 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.030303:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382382976:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.030378:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.032207:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35941:x1875453360512384:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 127617us (127861us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.032219:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4581 00000001:00080000:2.0:1788572547.032739:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970714 is committed 00000020:00080000:2.0:1788572547.032757:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2f64d80 refcount 2 committed 0 result 0 batchid 135 00000020:00080000:2.0:1788572547.032762:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c687c500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2f640c0 00000020:00080000:2.0:1788572547.032765:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c687c700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ca982800 00000020:00080000:2.0:1788572547.032784:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 132 batchid updated 00000001:00080000:2.0:1788572547.032791:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970716 is committed 00000001:00080000:2.0:1788572547.032825:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970717 is committed 00000020:00080000:2.0:1788572547.032853:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627dffb3780 refcount 2 committed 0 result 0 batchid 134 00000020:00080000:2.0:1788572547.032857:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceaf00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627dffb3a80 00000020:00080000:2.0:1788572547.032861:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cead00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627dfe8c400 00000020:00080000:2.0:1788572547.032877:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ff3fee40 refcount 2 committed 0 result 0 batchid 133 00000020:00080000:2.0:1788572547.032880:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cea500 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff3fe0c0 00000020:00080000:2.0:1788572547.032883:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceab00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627dff77800 00000001:00080000:2.0:1788572547.032896:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967644 is committed 00000001:00080000:2.0:1788572547.032904:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967647 is committed 00000001:00080000:2.0:1788572547.032921:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967645 is committed 00000020:00080000:2.0:1788572547.032939:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 135 committed batchid 132 00000020:00080000:2.0:1788572547.033131:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 132 00000100:00100000:2.0:1788572547.037785:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.037884:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514816 00000100:00100000:0.0:1788572547.037895:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514816 00000100:00100000:0.0:1788572547.037955:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1285 00000100:00100000:0.0:1788572547.037967:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360514816:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.038099:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360514816:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 133us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.038111:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1285 00000100:00100000:1.0:1788572547.038303:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382382976:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572547.038331:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/1,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.038560:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360513664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 83186us (83431us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.038567:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1281 00000100:00100000:2.0:1788572547.040418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.040529:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360514944 00000100:00100000:1.0:1788572547.040537:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360514944 00010000:00010000:1.0:1788572547.040576:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.040616:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1085 00000100:00100000:1.0:1788572547.040625:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360514944:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.040638:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.040660:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.040672:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.040695:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007785cc1b/0x85ace67feb1640b8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0385 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.040711:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.040715:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.040721:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.040730:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.040765:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.040812:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360514944:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 186us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.040820:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1085 00010000:00010000:2.0:1788572547.041597:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041609:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041635:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041644:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041652:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041665:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572547.041670:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041679:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041694:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.041702:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb1640cd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.041778:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360514688:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11643us (11777us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.041786:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1284 00000100:00100000:2.0:1788572547.045150:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.045190:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.045278:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360515200 00000100:00100000:0.0:1788572547.045289:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360515200 00000100:00100000:0.0:1788572547.045351:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360515456 00000100:00100000:0.0:1788572547.045354:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360515456 00000100:00100000:0.0:1788572547.045375:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4587 00000100:00100000:0.0:1788572547.045384:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35941:x1875453360515200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.045457:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.045486:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb164064 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0346 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.045506:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:0.0:1788572547.045511:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572547.045539:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.045553:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.045564:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572547.045724:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bffdd013 reply state 00000000588fe546 transno 4294970719 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.045738:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572547.045832:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35941:x1875453360515200:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 448us (691us total) trans 4294970719 rc 0/0 00000100:00100000:0.0:1788572547.045840:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4587 00000100:00100000:0.0:1788572547.045859:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4588 00000100:00100000:0.0:1788572547.045866:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35942:x1875453360515456:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.045886:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.045898:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164087 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f035b expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.045908:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:0.0:1788572547.045911:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572547.045923:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.045932:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.045939:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572547.046036:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7544ddd reply state 00000000bc12ac1f transno 4294970720 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.046044:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572547.046083:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35942:x1875453360515456:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 217us (891us total) trans 4294970720 rc 0/0 00000100:00100000:0.0:1788572547.046092:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4588 00000100:00100000:1.0:1788572547.051136:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572547.051397:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.051410:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.051418:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.051429:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.051434:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.051452:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.051469:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:2.0:1788572547.051473:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.051478:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.051484:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640db lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.051670:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360515584 00000100:00100000:3.0:1788572547.051680:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360515584 00000100:00100000:3.0:1788572547.051728:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1286 00000100:00100000:3.0:1788572547.051738:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360515584:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572547.051975:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x55:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572547.051979:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x55:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:3.0:1788572547.051982:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x55:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:3.0:1788572547.051984:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x55:0x0] updating lvb ctime from disk: 1788572546 -> 1788572526 00002000:00010000:3.0:1788572547.051986:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x55:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572547.052072:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360515584:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 335us (944us total) trans 4294967649 rc 0/0 00000100:00100000:3.0:1788572547.052080:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1286 00000100:00100000:1.0:1788572547.054224:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.054326:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360515712 00000100:00100000:3.0:1788572547.054335:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360515712 00000100:00100000:3.0:1788572547.054395:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1287 00000100:00100000:3.0:1788572547.054405:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360515712:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572547.054658:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572547.054662:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb mtime from disk: 0 -> 1788572518 00002000:00010000:3.0:1788572547.054664:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb atime from disk: 0 -> 1788572518 00002000:00010000:3.0:1788572547.054666:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb ctime from disk: 1788572546 -> 1788572526 00002000:00010000:3.0:1788572547.054669:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572547.054729:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360515712:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 325us (514us total) trans 4294967648 rc 0/0 00000100:00100000:3.0:1788572547.054737:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1287 00010000:00010000:0.0:1788572547.055790:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.055805:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.055813:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.055819:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.055824:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.055832:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.055849:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:0.0:1788572547.055852:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.055858:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.055864:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640d4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.058162:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.058787:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360515968 00000100:00100000:0.0:1788572547.058798:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360515968 00000100:00100000:0.0:1788572547.058858:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4589 00000100:00100000:0.0:1788572547.058869:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35941:x1875453360515968:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.058952:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.058971:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.058980:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.059047:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.059061:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 64 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.059076:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382383104:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.059152:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d540 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.062088:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.062255:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382383104:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.062284:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.062631:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360516224 00000100:00100000:0.0:1788572547.062640:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360516224 00000100:00100000:0.0:1788572547.062698:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4590 00000100:00100000:0.0:1788572547.062708:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35942:x1875453360516224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.062773:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.062788:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.062796:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.062834:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.062855:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 65 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.062871:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382383232:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.062953:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d900 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.070769:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.070860:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.070976:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382383232:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.071023:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.071184:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360516352 00000100:00100000:0.0:1788572547.071204:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360516352 00010000:00010000:0.0:1788572547.071245:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.071287:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1086 00000100:00100000:0.0:1788572547.071296:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360516352:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.071312:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.071319:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071331:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071353:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb163fa0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0307 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071367:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:0.0:1788572547.071373:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071383:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071391:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071415:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.071479:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360516352:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 183us (720us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.071486:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1086 00010000:00010000:0.0:1788572547.071511:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071518:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071547:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071554:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071561:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071567:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.071571:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071584:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071593:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:0.0:1788572547.071595:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572547.071599:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.071605:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1640e2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.071644:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35941:x1875453360515968:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 12776us (13489us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572547.071654:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4589 00000100:00100000:2.0:1788572547.071808:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.071969:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360516736 00000100:00100000:3.0:1788572547.071974:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360516736 00010000:00010000:3.0:1788572547.071993:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 64 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.072038:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1087 00000100:00100000:3.0:1788572547.072047:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360516736:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.072055:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.072061:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 64 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072070:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 64 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072083:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16402c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0331 expref: 64 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072095:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572547.072099:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072107:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072114:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072132:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.072156:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2349:x1875453360516736:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 111us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.072162:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1087 00010000:00010000:3.0:1788572547.072199:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072206:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072221:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072227:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072232:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072237:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.072240:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072246:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072255:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572547.072258:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572547.072261:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.072266:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1640e9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.072301:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35942:x1875453360516224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 9595us (10228us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572547.072308:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4590 00000100:00100000:0.0:1788572547.076675:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360516608 00000100:00100000:0.0:1788572547.076691:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360516608 00000100:00100000:0.0:1788572547.076757:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4591 00000100:00100000:0.0:1788572547.076770:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35943:x1875453360516608:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.076874:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.076894:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.076907:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.077201:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.077218:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.077227:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.077264:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.077277:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 63 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.077291:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382383360:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.078376:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.078531:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.078677:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.085321:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.085463:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382383360:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.085493:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.085609:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360516864 00000100:00100000:0.0:1788572547.085618:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360516864 00000100:00100000:0.0:1788572547.085657:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360517120 00000100:00100000:0.0:1788572547.085661:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360517120 00010000:00010000:0.0:1788572547.085696:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 64 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.085737:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1088 00000100:00100000:0.0:1788572547.085748:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360517120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.085764:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.085771:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 64 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.085790:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 64 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.085811:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164072 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f034d expref: 64 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.085827:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:0.0:1788572547.085832:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.085842:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.085865:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.085893:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.085949:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2350:x1875453360517120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 202us (638us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.085957:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1088 00010000:00010000:3.0:1788572547.086089:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.086098:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572547.086297:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970721 00000001:00080000:3.0:1788572547.086472:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970722 00000001:00080000:3.0:1788572547.086519:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970723 00000100:00100000:3.0:1788572547.086604:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bbad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382383488:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572547.086655:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572547.086690:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c978d780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.087274:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382383488 00000100:00100000:0.0:1788572547.087281:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382383488 00000100:00100000:0.0:1788572547.087328:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:0.0:1788572547.087336:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382383488:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572547.087416:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382383488:12345-0@lo:1000:mdt00_000.0 Request processed in 80us (760us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.087424:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:0.0:1788572547.087473:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967643 gen 2 00000004:00080000:0.0:1788572547.087479:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967643 committed transno 4294967643 00000100:00100000:0.0:1788572547.087499:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bbad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382383488:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1788572547.088083:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970724 00000100:00100000:0.0:1788572547.088153:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382383616:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572547.088406:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572547.088650:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.089459:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360516992 00000100:00100000:0.0:1788572547.089467:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360516992 00000100:00100000:0.0:1788572547.089498:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1288 00000100:00100000:0.0:1788572547.089507:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360516864:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.089550:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/1,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.089571:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.089581:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x55:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1788572547.089586:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.089591:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 4/1,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.089622:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.089635:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 12 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.089653:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382383744:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.089725:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.090252:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1289 00000100:00100000:2.0:1788572547.090270:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360516992:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.090303:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.090322:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.090328:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x56:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1788572547.090331:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572547.090335:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.090359:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.090368:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 11 pid: 6682 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.090379:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382383872:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572547.090417:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077180 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.090988:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382383616 00000100:00100000:3.0:1788572547.090996:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382383616 00000100:00100000:2.0:1788572547.094936:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.094995:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.095150:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382383744:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.095179:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/1,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.095827:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360517376 00000100:00100000:2.0:1788572547.095838:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360517376 00010000:00010000:2.0:1788572547.095892:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 12 pid: 6682 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.095938:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360517248 00000100:00100000:2.0:1788572547.095942:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360517248 00010000:00010000:2.0:1788572547.095957:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.095980:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1089 00000100:00100000:2.0:1788572547.095991:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360517376:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572547.096031:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.096054:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 12 pid: 6682 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.096065:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 12 pid: 6682 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.096091:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572547.096096:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572547.096102:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.096113:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.096135:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.096351:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1090 00000100:00100000:0.0:1788572547.096363:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360517248:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.096372:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.096388:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096400:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096414:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bac2935f/0x85ace67feb16408e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x55:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0362 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096426:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.096429:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.096433:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096456:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096474:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.096496:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360517248:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 136us (1501us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.096503:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1090 00010000:00010000:0.0:1788572547.096531:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096540:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096563:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096572:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096580:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096588:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572547.096591:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096606:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096617:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 2/0,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.096626:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb1640fe lrc: 0/0,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.096689:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360516864:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 7183us (18161us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.096697:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1288 00000100:00100000:2.0:1788572547.096801:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360517376:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 202us (1265us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.096810:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1089 00000100:00100000:3.0:1788572547.098238:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:3.0:1788572547.098250:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382383616:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572547.098555:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382383616:12345-0@lo:1000:mdt00_000.0 Request processed in 306us (10148us total) trans 4294967650 rc 0/0 00000100:00100000:3.0:1788572547.098562:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:2.0:1788572547.099741:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382383616:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1788572547.099791:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.099803:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.099811:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.099817:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.099823:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.099831:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.099858:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:2.0:1788572547.099862:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.099868:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.099875:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1640f7 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100274:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100283:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100289:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100294:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627d21fe000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.100297:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100303:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100312:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9627fb1e7800) 00010000:00010000:2.0:1788572547.100314:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.100317:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100323:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627d21fe000/0x85ace67feb15fe5b lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100349:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100364:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100375:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100383:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100388:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100395:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100399:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.100402:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100407:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100414:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9627fb1e7800) 00010000:00010000:2.0:1788572547.100417:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.100420:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100426:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb16410c lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100445:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100453:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100458:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100463:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.100466:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100472:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100479:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9627fb1e7800) 00010000:00010000:2.0:1788572547.100482:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572547.100485:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.100491:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1640f0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.100726:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35943:x1875453360516608:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 23955us (29874us total) trans 4294970725 rc 0/0 00000100:00100000:2.0:1788572547.100738:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4591 00000020:00080000:2.0:1788572547.100744:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00010000:00010000:1.0:1788572547.101083:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb1640aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x56:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0377 expref: 11 pid: 6682 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.101098:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382383872:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572547.101119:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101130:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101136:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101165:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101172:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101178:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101184:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc14d24f0) 00000100:00100000:3.0:1788572547.101339:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572547.101488:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101499:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101515:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101524:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048ed1922/0x85ace67feb164105 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.101642:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360516992:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11380us (22963us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.101651:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1289 00000100:00100000:1.0:1788572547.101679:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360517632 00000100:00100000:1.0:1788572547.101686:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360517632 00000100:00100000:1.0:1788572547.101726:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4592 00000100:00100000:1.0:1788572547.101734:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35941:x1875453360517632:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.101805:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101818:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.101838:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.102867:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.102886:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.102895:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 4/0,1 mode: --/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.102938:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 67 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.102952:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 67 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.102967:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7559500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382384000:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.103049:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.106280:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.106397:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360517888 00000100:00100000:1.0:1788572547.106409:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360517888 00000100:00100000:1.0:1788572547.106475:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 959 00000100:00100000:1.0:1788572547.106486:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35943:x1875453360517888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572547.107350:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x9e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572547.107470:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970727 00010000:00010000:1.0:1788572547.107523:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.107541:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.107553:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.107561:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.107567:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.107577:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.107593:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572547.107837:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35943:x1875453360517888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1350us (1564us total) trans 4294970728 rc 0/0 00000100:00100000:1.0:1788572547.107856:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 959 00000020:00080000:1.0:1788572547.107861:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572547.107993:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572547.108001:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.108033:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572547.108037:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.108045:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.108058:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9e:0x0].0x0 (ffff9627d1184000) 00010000:00010000:3.0:1788572547.108061:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572547.108066:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.108072:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164121 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.108110:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572547.108113:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572547.108115:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572547.115640:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.115672:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.115708:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360518016 00000100:00100000:2.0:1788572547.115718:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360518016 00010000:00010000:2.0:1788572547.115764:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 70 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.115803:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1091 00000100:00100000:2.0:1788572547.115814:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360518016:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572547.115831:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.115839:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 70 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.115866:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 70 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.115889:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:2.0:1788572547.115895:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.115914:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.115921:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.115941:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572547.115985:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360518016:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 173us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.115993:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1091 00000100:00100000:2.0:1788572547.116039:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360518272 00000100:00100000:2.0:1788572547.116043:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360518272 00000100:00100000:2.0:1788572547.116068:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4593 00000100:00100000:2.0:1788572547.116075:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:35942:x1875453360518272:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.116153:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116169:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116181:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116449:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116461:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116474:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 4/0,1 mode: --/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116509:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116523:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 71 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.116538:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382384128:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572547.116596:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13170c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788572547.116640:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164002 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0323 expref: 71 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.116657:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7559500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382384000:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572547.116677:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116686:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.116692:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572547.116783:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970726 00000001:00080000:2.0:1788572547.116922:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970730 00000001:00080000:2.0:1788572547.116968:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970731 00000100:00100000:2.0:1788572547.117061:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382384256:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572547.117103:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.117142:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382384256 00000100:00100000:2.0:1788572547.117146:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382384256 00000100:00100000:2.0:1788572547.117178:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1788572547.117186:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382384256:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572547.117244:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382384256:12345-0@lo:1000:mdt00_003.0 Request processed in 59us (140us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.117251:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1788572547.117271:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2463000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.117290:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382384256:0@lo:1000:mdt00_003.0 00000001:00080000:2.0:1788572547.117638:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970732 00000100:00100000:2.0:1788572547.117687:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382384384:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572547.117714:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.117740:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382384384 00000100:00100000:2.0:1788572547.117744:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382384384 00000100:00100000:2.0:1788572547.117769:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1788572547.117776:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382384384:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572547.117916:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:9040:x1875453382384384:12345-0@lo:1000:mdt00_003.0 Request processed in 140us (200us total) trans 4294967651 rc 0/0 00000100:00100000:2.0:1788572547.117923:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1788572547.117944:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.117964:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382384384:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1788572547.117991:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.117999:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118028:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118033:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.118038:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118045:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118057:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9626c1d8f200) 00010000:00010000:2.0:1788572547.118060:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.118065:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118070:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16411a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118109:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118115:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118119:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118124:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.118127:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118136:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118144:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:2.0:1788572547.118146:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.118149:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118154:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb15fd3c lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118196:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118206:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118215:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118221:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118226:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118231:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118235:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.118238:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118244:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118251:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:2.0:1788572547.118253:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.118256:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118261:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164136 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118271:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118276:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118281:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118286:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.118288:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118293:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118300:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:2.0:1788572547.118302:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572547.118304:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.118313:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164113 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.118420:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35941:x1875453360517632:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 16684us (17079us total) trans 4294970733 rc 0/0 00000100:00100000:2.0:1788572547.118428:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4592 00000020:00080000:2.0:1788572547.118432:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572547.122749:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.122922:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360518528 00000100:00100000:0.0:1788572547.122933:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360518528 00000100:00100000:0.0:1788572547.122995:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 960 00000100:00100000:0.0:1788572547.123031:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:35941:x1875453360518528:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572547.123425:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x9c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572547.123483:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970734 00010000:00010000:0.0:1788572547.123527:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.123550:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.123562:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.123575:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.123582:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.123588:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.123606:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572547.123699:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:35941:x1875453360518528:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 695us (961us total) trans 4294970735 rc 0/0 00000100:00100000:0.0:1788572547.123707:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 960 00000020:00080000:0.0:1788572547.123712:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788572547.125163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.125813:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360518656 00000100:00100000:0.0:1788572547.125822:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360518656 00010000:00010000:0.0:1788572547.125869:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 77 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.125919:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1092 00000100:00100000:0.0:1788572547.125929:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:2349:x1875453360518656:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.125943:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.125949:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 77 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.125963:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 77 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.125989:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:0.0:1788572547.125994:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.126028:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.126036:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.126053:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.126090:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:2349:x1875453360518656:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 164us (933us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.126097:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1092 00010000:00010000:2.0:1788572547.126588:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164048 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0338 expref: 75 pid: 9040 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.126606:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382384128:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572547.126627:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.126637:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.126643:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572547.126792:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970729 00000001:00080000:2.0:1788572547.126943:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970736 00000001:00080000:2.0:1788572547.126982:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970737 00000100:00100000:2.0:1788572547.127070:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382384512:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572547.127116:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.127142:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9fae40 going to sleep for 11 seconds 00010000:00010000:2.0:1788572547.127158:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572547.127165:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.127173:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572547.127178:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.127290:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.127304:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9c:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:2.0:1788572547.127307:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572547.127311:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 1/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.127318:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16413d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.127334:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.127335:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.127337:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572547.127565:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382384512 00000100:00100000:0.0:1788572547.127572:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382384512 00000100:00100000:0.0:1788572547.127616:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:0.0:1788572547.127624:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382384512:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572547.127717:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382384512:12345-0@lo:1000:mdt00_002.0 Request processed in 94us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.127724:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:1.0:1788572547.127817:0:5839:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967645 gen 2 00000004:00080000:1.0:1788572547.127824:0:5839:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967644 committed transno 4294967645 00000004:00080000:1.0:1788572547.127840:0:5839:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967645 committed transno 4294967645 00000100:00100000:1.0:1788572547.127860:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382384512:0@lo:1000:mdt00_002.0 00000001:00080000:1.0:1788572547.128418:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970738 00000100:00100000:1.0:1788572547.128494:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382384640:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1788572547.128536:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.128572:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.128697:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382384640 00000100:00100000:2.0:1788572547.128703:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382384640 00000100:00100000:2.0:1788572547.128743:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:2.0:1788572547.128750:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382384640:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572547.129054:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382384640:12345-0@lo:1000:mdt00_002.0 Request processed in 303us (516us total) trans 4294967649 rc 0/0 00000100:00100000:2.0:1788572547.129061:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:2.0:1788572547.129100:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382384640:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1788572547.129132:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129145:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129151:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129157:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.129162:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129170:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129184:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:2.0:1788572547.129317:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.129323:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129330:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16412f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129360:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129366:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129372:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129377:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.129381:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129387:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129396:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627ffc16700) 00010000:00010000:2.0:1788572547.129398:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.129401:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129407:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb15fe00 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129429:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129452:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129463:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129470:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129476:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129481:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129485:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.129488:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129493:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129499:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627ffc16700) 00010000:00010000:2.0:1788572547.129502:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.129504:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129509:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb164144 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129518:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129524:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129528:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129533:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.129535:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129540:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129547:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627ffc16700) 00010000:00010000:2.0:1788572547.129549:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572547.129552:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.129557:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164128 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.129763:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35942:x1875453360518272:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 13685us (14088us total) trans 4294970739 rc 0/0 00000100:00100000:2.0:1788572547.129773:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4593 00000020:00080000:2.0:1788572547.129777:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:2.0:1788572547.132798:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572547.132810:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572547.133170:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 163 count 101 00000040:00080000:2.0:1788572547.133178:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:2.0:1788572547.133396:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5bb80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382384768:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572547.133538:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2faae40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.136221:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.136354:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360518912 00000100:00100000:1.0:1788572547.136365:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360518912 00000100:00100000:1.0:1788572547.136424:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 961 00000100:00100000:1.0:1788572547.136445:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35942:x1875453360518912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572547.136834:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x9d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572547.136881:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970740 00010000:00010000:1.0:1788572547.136933:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.136955:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.136967:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.136975:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.136981:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.136987:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.137185:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572547.137294:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35942:x1875453360518912:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 859us (1079us total) trans 4294970741 rc 0/0 00000100:00100000:1.0:1788572547.137301:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 961 00000020:00080000:1.0:1788572547.137306:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572547.137359:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572547.137367:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.137374:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572547.137379:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.137386:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.137399:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9d:0x0].0x0 (ffff9627d1184000) 00010000:00010000:2.0:1788572547.137401:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572547.137407:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 1/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.137413:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16414b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.137431:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.137433:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.137434:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572547.143500:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.143660:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5bb80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382384768:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572547.143920:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519040 00000100:00100000:2.0:1788572547.143927:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519040 00000100:00100000:2.0:1788572547.143988:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4594 00000100:00100000:2.0:1788572547.143998:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35945:x1875453360519040:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572547.144062:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa7:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572547.144177:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35945:x1875453360519040:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 180us (687us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.144185:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4594 00000100:00100000:0.0:1788572547.150553:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.150592:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.150627:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.150701:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519168 00000100:00100000:3.0:1788572547.150712:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519168 00000100:00100000:3.0:1788572547.150773:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519296 00000100:00100000:3.0:1788572547.150777:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519296 00000100:00100000:3.0:1788572547.150796:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519424 00000100:00100000:3.0:1788572547.150799:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519424 00000100:00100000:3.0:1788572547.150818:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4595 00000100:00100000:3.0:1788572547.150827:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35944:x1875453360519168:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572547.150862:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xab:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572547.150983:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35944:x1875453360519168:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 156us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.150990:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4595 00000100:00100000:3.0:1788572547.151036:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4596 00000100:00100000:3.0:1788572547.151044:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35946:x1875453360519296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572547.151060:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xac:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572547.151104:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35946:x1875453360519296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 61us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.151110:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4596 00000100:00100000:3.0:1788572547.151122:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4597 00000100:00100000:3.0:1788572547.151128:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35945:x1875453360519424:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.151185:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151203:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151216:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151234:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151242:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151249:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151273:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151281:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151287:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.151315:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa7:0x0]: returning size 3 00000004:00200000:3.0:1788572547.151319:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.151333:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151344:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151351:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151358:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151364:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.151368:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151376:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151388:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572547.151391:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.151396:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151402:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164152 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151415:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151421:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151426:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151430:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.151432:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151449:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151457:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xd331be57 (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572547.151459:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.151462:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151471:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164159 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xd331be57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151485:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151490:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151495:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151499:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.151502:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151507:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151514:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627c2abcb00) 00010000:00010000:3.0:1788572547.151516:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.151519:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.151524:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164160 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.151571:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35945:x1875453360519424:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 442us (941us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.151578:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4597 00000100:00100000:0.0:1788572547.158535:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.158681:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519552 00000100:00100000:3.0:1788572547.158691:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519552 00000100:00100000:3.0:1788572547.158758:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4598 00000100:00100000:3.0:1788572547.158769:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35946:x1875453360519552:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.158852:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158870:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158882:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158897:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158904:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158923:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158947:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158956:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.158963:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.159041:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xac:0x0]: returning size 3 00000004:00200000:3.0:1788572547.159046:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.159061:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159073:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159079:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159086:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159092:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.159097:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159105:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159117:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572547.159120:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.159125:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159131:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164167 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159142:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159149:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159159:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159163:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.159166:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159172:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159179:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x2df001ca (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572547.159182:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.159340:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159349:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb16416e lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2df001ca bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159366:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159373:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159378:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159383:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.159386:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159395:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159403:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:3.0:1788572547.159407:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.159410:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.159415:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164175 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.159500:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35946:x1875453360519552:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 731us (974us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.159508:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4598 00000100:00100000:1.0:1788572547.161650:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.161720:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519808 00000100:00100000:3.0:1788572547.161729:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519808 00000100:00100000:3.0:1788572547.161788:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4599 00000100:00100000:3.0:1788572547.161797:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35945:x1875453360519808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.161815:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.161833:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16417c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f038c expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.161952:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.161967:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.161979:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.161990:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa7:0x0]: returning size 3 00010000:00010000:3.0:1788572547.162129:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162137:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162144:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162150:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.162154:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162161:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162174:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627c2abcb00) 00010000:00010000:3.0:1788572547.162176:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.162181:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162198:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164183 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162241:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16417c lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f038c expref: 82 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162252:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572547.162254:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572547.162258:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16417c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f038c expref: 82 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.162267:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572547.162317:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35945:x1875453360519808:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 521us (676us total) trans 4294970742 rc 301/301 00000100:00100000:3.0:1788572547.162324:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4599 00000100:00100000:1.0:1788572547.167676:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.167780:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.168752:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.169504:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360520320 00000100:00100000:0.0:1788572547.169516:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360520320 00000100:00100000:0.0:1788572547.169584:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360519936 00000100:00100000:0.0:1788572547.169588:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360519936 00000100:00100000:0.0:1788572547.169615:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4600 00000100:00100000:0.0:1788572547.169625:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35946:x1875453360520320:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.169656:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.169681:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164191 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0393 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.169793:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.169816:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.169830:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.169844:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xac:0x0]: returning size 3 00010000:00010000:0.0:1788572547.169982:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.169990:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.169999:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170027:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.170031:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170040:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170054:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:0.0:1788572547.170057:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572547.170061:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170068:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164198 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170114:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164191 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0393 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170124:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572547.170126:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572547.170130:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164191 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0393 expref: 84 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170139:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd400, rc 0) 00000100:00100000:0.0:1788572547.170207:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35946:x1875453360520320:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 583us (2544us total) trans 4294970743 rc 301/301 00000100:00100000:0.0:1788572547.170214:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4600 00000100:00100000:0.0:1788572547.170236:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4601 00000100:00100000:0.0:1788572547.170245:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35944:x1875453360519936:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.170279:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170290:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170302:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170316:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170323:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170329:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170351:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170358:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170363:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.170387:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xab:0x0]: returning size 3 00000004:00200000:0.0:1788572547.170391:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572547.170399:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170407:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170413:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170421:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170426:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.170428:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170433:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170460:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572547.170463:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.170465:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170471:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1641a6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170480:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170486:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170491:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170495:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.170498:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170506:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170512:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xc5132acb (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572547.170514:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.170517:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170522:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1641ad lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xc5132acb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170535:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170540:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170545:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170550:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.170552:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170558:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170563:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9627c2abc000) 00010000:00010000:0.0:1788572547.170566:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.170568:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.170577:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1641b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.170612:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35944:x1875453360519936:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 368us (2831us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.170619:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4601 00000100:00100000:1.0:1788572547.174884:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.174989:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360520064 00000100:00100000:3.0:1788572547.174999:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360520064 00000100:00100000:3.0:1788572547.175078:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360520448 00000100:00100000:3.0:1788572547.175082:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360520448 00000100:00100000:3.0:1788572547.175107:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4602 00000100:00100000:3.0:1788572547.175116:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35945:x1875453360520064:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572547.175146:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa7:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572547.175273:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35945:x1875453360520064:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 157us (6523us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.175280:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4602 00000100:00100000:3.0:1788572547.175296:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4603 00000100:00100000:3.0:1788572547.175303:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35946:x1875453360520448:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572547.175317:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xac:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572547.175356:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35946:x1875453360520448:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 54us (483us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.175362:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4603 00000100:00100000:0.0:1788572547.179160:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.179283:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360520704 00000100:00100000:3.0:1788572547.179294:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360520704 00000100:00100000:3.0:1788572547.179355:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4604 00000100:00100000:3.0:1788572547.179367:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35944:x1875453360520704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.179388:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.179410:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1641bb lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f039a expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179530:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179546:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179558:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.179571:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xab:0x0]: returning size 3 00010000:00010000:3.0:1788572547.179673:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179682:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179689:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179694:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.179698:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179706:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179718:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9627c2abc000) 00010000:00010000:3.0:1788572547.179722:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.179726:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179733:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb1641c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179777:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1641bb lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f039a expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179786:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572547.179788:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572547.179791:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1641bb lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f039a expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.179800:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572547.179855:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35944:x1875453360520704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 491us (705us total) trans 4294970744 rc 301/301 00000100:00100000:3.0:1788572547.179862:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4604 00000100:00100000:0.0:1788572547.183790:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.184689:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.184765:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360520832 00000100:00100000:0.0:1788572547.184775:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360520832 00000100:00100000:0.0:1788572547.184838:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360520960 00000100:00100000:0.0:1788572547.184841:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360520960 00000100:00100000:0.0:1788572547.184873:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4605 00000100:00100000:0.0:1788572547.184883:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35945:x1875453360520832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.184912:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.184934:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1641d0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03a1 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185025:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185040:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185051:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185068:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185076:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185082:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185429:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1641e5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185451:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1641e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185458:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1641e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.185488:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:0.0:1788572547.185491:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572547.185510:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1641e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185523:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185530:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185536:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185542:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.185546:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185555:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185568:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572547.185571:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.185577:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185583:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb1641d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185595:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185600:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185606:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185611:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.185614:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185621:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185628:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9627fb1e7300) 00010000:00010000:0.0:1788572547.185631:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.185633:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185640:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb1641de lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185673:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1641d0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03a1 expref: 86 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185687:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1641e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03a1 expref: 85 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.185693:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.185695:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572547.185698:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc 0) 00000100:00100000:0.0:1788572547.185765:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35945:x1875453360520832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 883us (1988us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.185773:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4605 00000100:00100000:0.0:1788572547.185796:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4606 00000100:00100000:0.0:1788572547.185805:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35944:x1875453360520960:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572547.185830:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xab:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572547.185895:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35944:x1875453360520960:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 91us (1208us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.185901:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4606 00000100:00100000:1.0:1788572547.192433:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.192528:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360521216 00000100:00100000:0.0:1788572547.192539:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360521216 00000100:00100000:0.0:1788572547.192615:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4607 00000100:00100000:0.0:1788572547.192626:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35945:x1875453360521216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.192650:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.192654:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572547.192680:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1641e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03a1 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.192700:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.192705:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572547.192732:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb1641ec lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03a8 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572547.193314:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x57:0x0], index 0: 0 00010000:00010000:0.0:1788572547.193518:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1641f3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.193532:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1641f3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.193543:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1641f3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.193603:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb1641ec lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03a8 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.193618:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1641f3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03a8 expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.193626:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:0.0:1788572547.193629:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572547.193632:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23eee00, rc 0) 00000100:00100000:0.0:1788572547.193691:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35945:x1875453360521216:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1067us (1268us total) trans 4294970745 rc 0/0 00000100:00100000:0.0:1788572547.193699:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4607 00010000:00010000:0.0:1788572547.195587:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572547.198415:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.198475:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360521344 00000100:00100000:3.0:1788572547.198486:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360521344 00000100:00100000:3.0:1788572547.198553:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4608 00000100:00100000:3.0:1788572547.198564:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35946:x1875453360521344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.198585:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.198608:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164201 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03af expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.198687:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.198704:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.198717:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.198734:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.198741:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.198747:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199131:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164216 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199142:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164216 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199149:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164216 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.199176:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572547.199181:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.199200:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164216 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199213:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199224:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199230:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199236:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.199241:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199248:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199261:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572547.199264:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.199269:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199275:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164208 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199286:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199292:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199298:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199304:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.199307:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199312:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199320:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572547.199322:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.199325:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199331:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16420f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199363:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164201 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03af expref: 87 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199376:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164216 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03af expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.199383:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.199385:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.199388:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572547.199463:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35946:x1875453360521344:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 900us (1059us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.199470:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4608 00000100:00100000:1.0:1788572547.205265:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.205315:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.205329:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360521600 00000100:00100000:0.0:1788572547.205340:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360521600 00000100:00100000:0.0:1788572547.205399:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360521728 00000100:00100000:0.0:1788572547.205403:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360521728 00000100:00100000:0.0:1788572547.205443:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4609 00000100:00100000:0.0:1788572547.205454:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35945:x1875453360521600:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.205521:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788572547.205547:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1641f3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03a8 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205566:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:0.0:1788572547.205583:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164216 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03af expref: 86 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205592:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.205596:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572547.205621:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205636:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205646:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205662:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205670:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205676:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205968:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205977:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205984:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.205990:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.205995:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206038:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206049:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.206052:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.206056:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.206057:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4610 00010000:00010000:0.0:1788572547.206062:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16421d lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.206068:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35944:x1875453360521728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.206072:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206078:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206079:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.206083:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206088:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.206091:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206092:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16422b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03b6 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206098:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206104:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x7653fc9e (ffff9627fb1e7300) 00010000:00010000:0.0:1788572547.206107:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.206109:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.206114:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164224 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x7653fc9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206137:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206146:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206153:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206165:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206171:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206176:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.206192:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35945:x1875453360521600:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 740us (937us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572547.206200:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4609 00010000:00010000:1.0:1788572547.206366:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164240 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206376:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164240 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206384:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164240 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.206409:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:1.0:1788572547.206413:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572547.206430:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164240 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206506:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206512:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206519:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206524:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.206527:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206532:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206540:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572547.206542:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.206545:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206551:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb164232 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206559:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206575:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206580:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206585:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.206588:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206594:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206600:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9627d1184000) 00010000:00010000:1.0:1788572547.206603:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.206605:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206610:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164239 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206637:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16422b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03b6 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206651:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164240 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03b6 expref: 85 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.206658:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572547.206659:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572547.206663:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96c00, rc 0) 00000100:00100000:1.0:1788572547.206701:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35944:x1875453360521728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 636us (1385us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.206708:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4610 00000100:00100000:2.0:1788572547.213829:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.213932:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.214587:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360521984 00000100:00100000:3.0:1788572547.214598:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360521984 00000100:00100000:3.0:1788572547.214693:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360522112 00000100:00100000:3.0:1788572547.214698:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360522112 00000100:00100000:0.0:1788572547.214744:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4611 00000100:00100000:0.0:1788572547.214758:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35946:x1875453360521984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.214781:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.214803:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb164247 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03bd expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572547.215616:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x57:0x0], index 1: 0 00010000:00010000:0.0:1788572547.215815:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb16424e lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.215833:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb16424e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216123:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb16424e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216198:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb164247 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03bd expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216215:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb16424e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03bd expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216224:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:0.0:1788572547.216227:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572547.216231:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc600, rc 0) 00000100:00100000:0.0:1788572547.216312:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35946:x1875453360521984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1556us (2493us total) trans 4294970746 rc 0/0 00000100:00100000:0.0:1788572547.216320:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4611 00000100:00100000:0.0:1788572547.216348:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4612 00000100:00100000:0.0:1788572547.216358:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35945:x1875453360522112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.216370:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.216382:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16425c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03c4 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216404:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216413:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216420:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216509:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16425c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216522:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.216529:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:0.0:1788572547.216531:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572547.216533:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bde00, rc 0) 00000100:00100000:0.0:1788572547.216566:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35945:x1875453360522112:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 211us (2634us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.216573:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4612 00000100:00100000:2.0:1788572547.221472:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.221570:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.221602:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360522368 00000100:00100000:2.0:1788572547.221611:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360522368 00000100:00100000:2.0:1788572547.221663:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360522624 00000100:00100000:2.0:1788572547.221667:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360522624 00000100:00100000:2.0:1788572547.221688:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4613 00000100:00100000:2.0:1788572547.221697:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35946:x1875453360522368:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.221762:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572547.221792:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb16424e lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03bd expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221814:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627fb1e7e00) 00010000:00010000:2.0:1788572547.221831:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164240 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03b6 expref: 88 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221840:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572547.221844:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572547.221881:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221895:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221906:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221933:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221941:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.221947:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222185:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222193:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222203:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222209:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.222213:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222221:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222229:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572547.222232:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.222235:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222240:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb16426a lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222249:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222255:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222261:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222266:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.222268:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222278:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222285:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xd5f5171 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.222287:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.222290:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222296:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb164271 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0xd5f5171 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.222379:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35946:x1875453360522368:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 682us (917us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572547.222387:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4613 00000100:00100000:2.0:1788572547.222410:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4614 00000100:00100000:2.0:1788572547.222418:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35945:x1875453360522624:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.222486:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222496:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.222504:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572547.222827:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].75, 88 off14312 00000004:00010000:2.0:1788572547.222922:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000922cfeaa reply state 000000005b4d8bee transno 4294970747 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.222936:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572547.222996:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35945:x1875453360522624:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 578us (1424us total) trans 4294970747 rc 0/0 00000100:00100000:2.0:1788572547.223027:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4614 00000100:00100000:1.0:1788572547.227066:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.227095:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.227129:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572547.227155:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.227166:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.227173:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.227178:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.227183:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.227206:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.227224:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:1.0:1788572547.227227:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572547.227232:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.227238:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb164278 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.227624:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360523008 00000100:00100000:2.0:1788572547.227634:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360523008 00000100:00100000:2.0:1788572547.227681:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360522752 00000100:00100000:2.0:1788572547.227684:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360522752 00000100:00100000:2.0:1788572547.227696:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360523136 00000100:00100000:2.0:1788572547.227698:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360523136 00000100:00100000:2.0:1788572547.227713:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4615 00000100:00100000:2.0:1788572547.227723:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35944:x1875453360523008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.227744:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.227761:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16427f lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03d2 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572547.228264:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x58:0x0], index 0: 0 00010000:00010000:2.0:1788572547.228470:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb164286 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228484:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb164286 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228493:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb164286 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228555:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16427f lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d2 expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228577:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb164286 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d2 expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228585:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572547.228588:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572547.228590:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f200, rc 0) 00000100:00100000:2.0:1788572547.228657:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35944:x1875453360523008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 934us (1600us total) trans 4294970748 rc 0/0 00000100:00100000:2.0:1788572547.228665:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4615 00000100:00100000:2.0:1788572547.228692:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4616 00000100:00100000:2.0:1788572547.228701:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35946:x1875453360522752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.228711:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.228721:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb164294 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03cb expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228741:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228749:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228757:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228834:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb164294 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 92 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228853:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228860:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:2.0:1788572547.228862:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572547.228864:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:2.0:1788572547.228894:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35946:x1875453360522752:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 195us (1798us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.228900:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4616 00000100:00100000:2.0:1788572547.228916:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4617 00000100:00100000:2.0:1788572547.228924:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35945:x1875453360523136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.228934:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.228943:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03d9 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228972:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228983:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 91 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.228999:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:2.0:1788572547.229001:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.229025:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc 0) 00000100:00100000:2.0:1788572547.229046:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35945:x1875453360523136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 123us (1916us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.229052:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4617 00000100:00100000:2.0:1788572547.235665:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.235709:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.235800:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.236143:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360523520 00000100:00100000:1.0:1788572547.236155:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360523520 00000100:00100000:1.0:1788572547.236375:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1290 00000100:00100000:1.0:1788572547.236387:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360523520:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.236709:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360523520:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 323us (907us total) trans 4294967650 rc 0/0 00000100:00100000:1.0:1788572547.236717:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1290 00000100:00100000:1.0:1788572547.240156:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.241283:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360523904 00000100:00100000:1.0:1788572547.241296:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360523904 00000100:00100000:1.0:1788572547.241366:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 962 00000100:00100000:1.0:1788572547.241376:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35945:x1875453360523904:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.241808:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35945:x1875453360523904:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 432us (1660us total) trans 4294970749 rc 0/0 00000100:00100000:1.0:1788572547.241816:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 962 00000100:00100000:3.0:1788572547.242053:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360523392 00000100:00100000:3.0:1788572547.242060:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360523392 00000100:00100000:3.0:1788572547.242117:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360523648 00000100:00100000:3.0:1788572547.242121:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360523648 00000100:00100000:3.0:1788572547.242143:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4618 00000100:00100000:3.0:1788572547.242149:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35946:x1875453360523392:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.242802:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.242823:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.242836:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.243184:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4619 00000100:00100000:0.0:1788572547.243220:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35944:x1875453360523648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.243233:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.243247:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1642b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03e0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.243299:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.243309:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.243318:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.243332:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.243339:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.243345:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572547.244065:0:23058:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].73, 88 off14208 00000004:00010000:3.0:1788572547.244157:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000305a372e reply state 00000000ea111637 transno 4294970750 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.244166:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572547.246255:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35946:x1875453360523392:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 4032us (10526us total) trans 4294970750 rc 0/0 00000100:00100000:3.0:1788572547.246267:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4618 00010000:00010000:0.0:1788572547.246927:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1642c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.246946:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1642c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.246958:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1642c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.246987:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:0.0:1788572547.246991:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572547.247033:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1642c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247049:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247055:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247062:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247067:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.247071:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247079:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247092:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572547.247095:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.247100:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247107:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1642b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247119:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247126:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247131:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247136:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.247139:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247144:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247151:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9627fb1e7600) 00010000:00010000:0.0:1788572547.247153:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.247156:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247161:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1642be lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247212:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1642b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e0 expref: 96 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247227:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1642c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e0 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.247234:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.247236:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572547.247239:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bda00, rc 0) 00000100:00100000:0.0:1788572547.247286:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35944:x1875453360523648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 4082us (11576us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.247293:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4619 00000100:00100000:2.0:1788572547.252402:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.252470:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.252524:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.252560:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360524160 00000100:00100000:2.0:1788572547.252571:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360524160 00000100:00100000:2.0:1788572547.252631:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360524416 00000100:00100000:2.0:1788572547.252635:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360524416 00000100:00100000:2.0:1788572547.252653:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360524544 00000100:00100000:2.0:1788572547.252655:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360524544 00000100:00100000:2.0:1788572547.252675:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4620 00000100:00100000:2.0:1788572547.252685:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35945:x1875453360524160:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.252800:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35945:x1875453360524160:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 116us (407us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.252807:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4620 00000100:00100000:2.0:1788572547.252828:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4621 00000100:00100000:2.0:1788572547.252834:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35944:x1875453360524416:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.252866:0:23058:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572547.252891:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb164286 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d2 expref: 96 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.252976:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95b00) 00010000:00010000:2.0:1788572547.252997:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1642c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e0 expref: 95 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253033:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572547.253037:0:23058:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572547.253060:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253074:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253083:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253098:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253104:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253111:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253483:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253493:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253499:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253505:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.253509:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253517:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253526:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572547.253529:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.253532:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253538:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1642cc lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253546:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253551:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253557:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253562:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.253564:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253570:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253577:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x92fc0a2c (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.253579:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.253582:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253588:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb1642d3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x92fc0a2c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.253642:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35944:x1875453360524416:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 807us (1170us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572547.253649:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4621 00000100:00100000:2.0:1788572547.253667:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4622 00000100:00100000:2.0:1788572547.253676:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35946:x1875453360524544:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.253690:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.253701:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03e7 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253744:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253756:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.253770:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:2.0:1788572547.253773:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.253776:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad600, rc 0) 00000100:00100000:2.0:1788572547.253798:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35946:x1875453360524544:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 124us (1274us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.253804:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4622 00010000:00010000:0.0:1788572547.254044:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.254052:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.254057:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.254062:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.254065:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.254071:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.254080:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:0.0:1788572547.254082:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572547.254085:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.254090:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1642a9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.258348:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.258392:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.258487:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360524800 00000100:00100000:3.0:1788572547.258497:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360524800 00000100:00100000:3.0:1788572547.258556:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1291 00000100:00100000:3.0:1788572547.258566:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360524800:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.258894:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360524800:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 328us (500us total) trans 4294967652 rc 0/0 00000100:00100000:3.0:1788572547.258918:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1291 00000100:00100000:2.0:1788572547.259583:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360524672 00000100:00100000:2.0:1788572547.259590:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360524672 00000100:00100000:2.0:1788572547.259638:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4623 00000100:00100000:2.0:1788572547.259646:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35945:x1875453360524672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.259661:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.259679:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1642e1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03ee expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.259748:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1642e8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.259762:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1642e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.259772:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1642e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572547.259803:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572547.259822:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1642e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.259850:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1642e1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03ee expref: 95 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.259867:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1642e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03ee expref: 94 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.259876:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:2.0:1788572547.259879:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572547.259882:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05800, rc 0) 00000100:00100000:2.0:1788572547.259939:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35945:x1875453360524672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 294us (1601us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.259949:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4623 00000100:00100000:1.0:1788572547.262995:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.263296:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360524928 00000100:00100000:2.0:1788572547.263316:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360524928 00000100:00100000:2.0:1788572547.263380:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4624 00000100:00100000:2.0:1788572547.263390:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35944:x1875453360524928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.263412:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.263446:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1642ef lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03f5 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.263486:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.263500:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.263511:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.263608:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1642ef lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 96 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.263628:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 95 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.263637:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.263640:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572547.263644:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04600, rc 0) 00000100:00100000:2.0:1788572547.263698:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35944:x1875453360524928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 310us (712us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.263705:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4624 00000100:00100000:1.0:1788572547.268616:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.268695:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360525184 00000100:00100000:2.0:1788572547.268706:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360525184 00000100:00100000:2.0:1788572547.268768:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4625 00000100:00100000:2.0:1788572547.268779:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35944:x1875453360525184:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.268884:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.268901:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.268920:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572547.269354:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].76, 88 off14400 00000004:00010000:2.0:1788572547.269454:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000033174208 reply state 0000000009bf4d41 transno 4294970751 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.269462:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572547.269564:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35944:x1875453360525184:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 785us (956us total) trans 4294970751 rc 0/0 00000100:00100000:2.0:1788572547.269572:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4625 00000100:00100000:3.0:1788572547.273230:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.273272:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572547.273378:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.273390:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.273397:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.273403:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.273408:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.273416:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.273432:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.273462:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572547.273469:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.273475:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1642fd lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.273633:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360525312 00000100:00100000:0.0:1788572547.273645:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360525312 00000100:00100000:0.0:1788572547.273703:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1292 00000100:00100000:0.0:1788572547.273714:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360525312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.273732:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.273751:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f03fc expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572547.273780:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x57:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7382, blocks: 0x1 00010000:00010000:0.0:1788572547.273798:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=895s v=(0 0) 00010000:00010000:0.0:1788572547.273803:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.273812:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.273829:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.273841:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bca00, rc 0) 00000100:00100000:0.0:1788572547.273919:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360525312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 197us (689us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.273926:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1292 00000100:00100000:1.0:1788572547.274192:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360525568 00000100:00100000:1.0:1788572547.274198:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360525568 00000100:00100000:1.0:1788572547.274236:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 963 00000100:00100000:1.0:1788572547.274244:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35946:x1875453360525568:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.274614:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35946:x1875453360525568:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 370us (1340us total) trans 4294970752 rc 0/0 00000100:00100000:1.0:1788572547.274621:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 963 00000100:00100000:3.0:1788572547.277606:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.277692:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360525696 00000100:00100000:2.0:1788572547.277702:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360525696 00000100:00100000:2.0:1788572547.277765:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4626 00000100:00100000:2.0:1788572547.277776:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35944:x1875453360525696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.277800:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.277820:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0403 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.277880:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 98 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.277896:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 98 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.277935:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.277938:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.277941:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00000100:00100000:2.0:1788572547.278000:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35944:x1875453360525696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 225us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.278039:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4626 00000100:00100000:3.0:1788572547.281551:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.281643:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360525952 00000100:00100000:2.0:1788572547.281654:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360525952 00000100:00100000:2.0:1788572547.281727:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4627 00000100:00100000:2.0:1788572547.281739:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35945:x1875453360525952:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.281860:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35945:x1875453360525952:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 123us (319us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.281868:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4627 00000100:00100000:3.0:1788572547.285638:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.285685:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.285771:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360526080 00000100:00100000:1.0:1788572547.285783:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360526080 00000100:00100000:1.0:1788572547.285861:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1293 00000100:00100000:1.0:1788572547.285872:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360526080:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.286566:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360526080:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 689us (932us total) trans 4294967651 rc 0/0 00000100:00100000:1.0:1788572547.286575:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1293 00000100:00100000:2.0:1788572547.287064:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360526336 00000100:00100000:2.0:1788572547.287071:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360526336 00000100:00100000:2.0:1788572547.287115:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4628 00000100:00100000:2.0:1788572547.287124:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35946:x1875453360526336:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.287221:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35946:x1875453360526336:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 98us (1535us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.287228:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4628 00000100:00100000:3.0:1788572547.291356:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.291479:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360526720 00000100:00100000:0.0:1788572547.291490:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360526720 00000100:00100000:0.0:1788572547.291553:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 964 00000100:00100000:0.0:1788572547.291565:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35944:x1875453360526720:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.292064:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35944:x1875453360526720:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 499us (717us total) trans 4294970753 rc 0/0 00000100:00100000:0.0:1788572547.292073:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 964 00000100:00100000:3.0:1788572547.294993:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.295088:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.295212:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360526848 00000100:00100000:1.0:1788572547.295222:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360526848 00000100:00100000:1.0:1788572547.295285:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1294 00000100:00100000:1.0:1788572547.295296:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35945:x1875453360526848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.295316:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.295336:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0411 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.295368:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.295377:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.295381:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.295385:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.295422:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.295452:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 14 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.295469:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382384896:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.295551:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.295817:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360526464 00000100:00100000:2.0:1788572547.295822:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360526464 00000100:00100000:2.0:1788572547.295865:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4629 00000100:00100000:2.0:1788572547.295873:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35946:x1875453360526464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.295884:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.295898:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164319 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f040a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.295970:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164320 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.295983:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164320 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.295995:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164320 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572547.296066:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572547.296092:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164320 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.296118:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164319 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f040a expref: 101 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.296133:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164320 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f040a expref: 100 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.296142:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:2.0:1788572547.296145:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572547.296148:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05400, rc 0) 00000100:00100000:2.0:1788572547.296312:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35946:x1875453360526464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 439us (1327us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.296320:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4629 00000100:00100000:1.0:1788572547.299408:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.300546:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360527104 00000100:00100000:2.0:1788572547.300559:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360527104 00000100:00100000:2.0:1788572547.300618:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4630 00000100:00100000:2.0:1788572547.300628:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35944:x1875453360527104:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.300753:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35944:x1875453360527104:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 126us (1353us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.300761:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4630 00000100:00100000:1.0:1788572547.305285:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.305331:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.305420:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360527360 00000100:00100000:2.0:1788572547.305431:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360527360 00000100:00100000:2.0:1788572547.305504:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4631 00000100:00100000:2.0:1788572547.305515:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35944:x1875453360527360:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.305537:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.305557:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164327 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0418 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.305643:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16432e lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.305659:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16432e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.305671:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16432e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572547.305711:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572547.305731:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16432e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.305759:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164327 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0418 expref: 102 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.305776:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16432e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0418 expref: 101 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.305785:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.305788:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572547.305791:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572547.305860:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35944:x1875453360527360:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 336us (517us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.305867:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4631 00000100:00100000:0.0:1788572547.306049:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360527232 00000100:00100000:0.0:1788572547.306054:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360527232 00010000:00010000:0.0:1788572547.306078:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 15 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.306107:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1093 00000100:00100000:0.0:1788572547.306114:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360527232:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.306124:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.306142:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 15 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.306154:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 15 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.306172:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.306177:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.306185:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.306194:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.306229:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 15 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.306243:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382385024:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572547.306294:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d540 going to sleep for 11 seconds 00010000:00010000:3.0:1788572547.306699:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059d071c1/0x85ace67feb164304 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03fc expref: 15 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.306713:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382384896:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.306736:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.306746:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97400, rc 0) 00000100:00100000:3.0:1788572547.306785:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35945:x1875453360526848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 11491us (11697us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.306792:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1294 00000100:00100000:0.0:1788572547.310873:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382385024:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572547.310903:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.310977:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360527232:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 4862us (5700us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.310989:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1093 00000100:00100000:0.0:1788572547.314879:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.314995:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360527488 00000100:00100000:2.0:1788572547.315049:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360527488 00000100:00100000:2.0:1788572547.315110:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1295 00000100:00100000:2.0:1788572547.315122:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360527488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.315143:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.315164:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f041f expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572547.315206:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x57:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7382, blocks: 0x1 00010000:00010000:2.0:1788572547.315228:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=895s v=(0 0) 00010000:00010000:2.0:1788572547.315232:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572547.315243:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.315260:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.315279:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:2.0:1788572547.315337:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360527488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 217us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.315344:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1295 00000100:00100000:3.0:1788572547.319224:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.319276:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.319420:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360527744 00000100:00100000:2.0:1788572547.319432:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360527744 00000100:00100000:2.0:1788572547.319505:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1296 00000100:00100000:2.0:1788572547.319516:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360527744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.319533:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.319556:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0426 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572547.319584:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x58:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7382, blocks: 0x1 00010000:00010000:2.0:1788572547.319605:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=895s v=(0 0) 00010000:00010000:2.0:1788572547.319610:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572547.319620:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.319639:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 14 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.319654:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572547.319716:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360527744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 201us (502us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.319727:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1296 00000100:00100000:1.0:1788572547.320037:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360527616 00000100:00100000:1.0:1788572547.320043:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360527616 00002000:00100000:1.0:1788572547.320078:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x57:0x0] (0->2) req@ffff9626c2c48e00 x1875453360527616/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.320088:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2c48e00 with x1875453360527616 ext(0->2) 00002000:00010000:1.0:1788572547.320093:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2c48e00 00000100:00100000:1.0:1788572547.320120:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 288 00000100:00100000:1.0:1788572547.320128:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2347:x1875453360527616:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.320575:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2347:x1875453360527616:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 447us (1297us total) trans 4294967652 rc 0/0 00000100:00100000:1.0:1788572547.320583:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 288 00002000:00100000:1.0:1788572547.320590:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x57:0x0] (0->2) req@ffff9626c2c48e00 x1875453360527616/t4294967652(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/448 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.320597:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2c48e00 with x1875453360527616 ext(0->2) 00002000:00010000:1.0:1788572547.320601:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2c48e00 00000100:00100000:3.0:1788572547.326845:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.326948:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.327061:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528000 00000100:00100000:0.0:1788572547.327071:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528000 00000100:00100000:0.0:1788572547.327135:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528128 00000100:00100000:0.0:1788572547.327138:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528128 00000100:00100000:0.0:1788572547.327160:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4632 00000100:00100000:0.0:1788572547.327169:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35946:x1875453360528000:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.327288:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35946:x1875453360528000:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 119us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.327296:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4632 00000100:00100000:0.0:1788572547.327315:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4633 00000100:00100000:0.0:1788572547.327323:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35945:x1875453360528128:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.335989:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.336850:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528384 00000100:00100000:2.0:1788572547.336864:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528384 00000100:00100000:2.0:1788572547.336943:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4634 00000100:00100000:2.0:1788572547.336955:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35944:x1875453360528384:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.337108:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35944:x1875453360528384:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 155us (1130us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.337116:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4634 00000100:00100000:0.0:1788572547.345639:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.345714:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.346226:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528512 00000100:00100000:1.0:1788572547.346236:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528512 00000100:00100000:1.0:1788572547.346288:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528640 00000100:00100000:1.0:1788572547.346291:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528640 00000100:00100000:1.0:1788572547.346312:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1297 00000100:00100000:1.0:1788572547.346321:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35946:x1875453360528512:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.346340:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.346361:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f042d expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346392:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346403:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.346408:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.346414:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346458:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346472:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.346487:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382385152:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.346560:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.346579:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1298 00000100:00100000:1.0:1788572547.346587:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:35944:x1875453360528640:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.346595:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.346605:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0434 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346620:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346627:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.346630:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.346634:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346650:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.346660:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.346670:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382385280:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.346688:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.349742:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.349877:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382385152:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.350141:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 14 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.350169:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:0.0:1788572547.350237:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35946:x1875453360528512:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3915us (4604us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.350248:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1297 00000100:00100000:1.0:1788572547.352806:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382385280:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572547.352843:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 17 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.352864:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572547.352938:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:35944:x1875453360528640:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6350us (7222us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.352946:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1298 00000100:00100000:1.0:1788572547.354638:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.355422:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528768 00000100:00100000:2.0:1788572547.355433:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528768 00010000:00010000:2.0:1788572547.355484:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.355838:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360528896 00000100:00100000:2.0:1788572547.355842:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360528896 00010000:00010000:2.0:1788572547.355870:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.355999:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1094 00000100:00100000:1.0:1788572547.356041:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360528768:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.356057:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.356083:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.356096:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.356118:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000525813d1/0x85ace67feb164335 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f041f expref: 14 pid: 6682 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.356135:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.356140:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.356149:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.356159:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.358184:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1095 00010000:00010000:1.0:1788572547.358330:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 13 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.358354:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382385408:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572547.358451:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.359316:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2349:x1875453360528896:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.359334:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.359359:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.359374:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.359395:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb16433c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0426 expref: 17 pid: 13272 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.359411:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.359415:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.359422:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.359432:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.359473:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.359485:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382385536:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572547.359523:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.370990:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382385536:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572547.371044:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.371106:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360528896:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 11889us (16474us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.371116:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1095 00000100:00100000:1.0:1788572547.371779:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382385408:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572547.371792:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.371834:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360528768:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 15797us (22102us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.371842:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1094 00000100:00100000:2.0:1788572547.381550:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.381587:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.381706:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529152 00000100:00100000:1.0:1788572547.381718:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529152 00002000:00100000:1.0:1788572547.381770:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x57:0x0] (0->2) req@ffff9626c4b33100 x1875453360529152/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.381782:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b33100 with x1875453360529152 ext(0->2) 00002000:00010000:1.0:1788572547.381789:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4b33100 00000100:00100000:1.0:1788572547.381813:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529024 00000100:00100000:1.0:1788572547.381817:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529024 00002000:00100000:1.0:1788572547.381835:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x58:0x0] (0->2) req@ffff9626c4b31500 x1875453360529024/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.381844:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b31500 with x1875453360529024 ext(0->2) 00002000:00010000:1.0:1788572547.381847:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b31500 00000100:00100000:1.0:1788572547.381866:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 289 00000100:00100000:1.0:1788572547.381876:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360529152:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.382424:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360529152:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 547us (882us total) trans 4294967653 rc 0/0 00000100:00100000:1.0:1788572547.382443:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 289 00002000:00100000:1.0:1788572547.382451:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x57:0x0] (0->2) req@ffff9626c4b33100 x1875453360529152/t4294967653(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/448 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.382460:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b33100 with x1875453360529152 ext(0->2) 00002000:00010000:1.0:1788572547.382463:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4b33100 00000100:00100000:1.0:1788572547.382482:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 290 00000100:00100000:1.0:1788572547.382490:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360529024:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.382736:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360529024:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 246us (1147us total) trans 4294967653 rc 0/0 00000100:00100000:1.0:1788572547.382744:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 290 00002000:00100000:1.0:1788572547.382750:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x58:0x0] (0->2) req@ffff9626c4b31500 x1875453360529024/t4294967653(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/448 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.382759:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b31500 with x1875453360529024 ext(0->2) 00002000:00010000:1.0:1788572547.382763:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b31500 00000100:00100000:2.0:1788572547.386202:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.386241:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.386270:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529280 00000100:00100000:2.0:1788572547.386278:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529280 00000100:00100000:2.0:1788572547.386320:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529408 00000100:00100000:2.0:1788572547.386322:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529408 00000100:00100000:2.0:1788572547.386339:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4635 00000100:00100000:2.0:1788572547.386348:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec193450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35944:x1875453360529280:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.386480:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec193450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+103:35944:x1875453360529280:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 132us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.386487:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4635 00000100:00100000:2.0:1788572547.386505:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4636 00000100:00100000:2.0:1788572547.386512:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35946:x1875453360529408:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.386550:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35946:x1875453360529408:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 39us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.386556:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4636 00000100:00100000:2.0:1788572547.388819:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.388906:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529536 00000100:00100000:0.0:1788572547.388924:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529536 00000100:00100000:0.0:1788572547.388974:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1299 00000100:00100000:0.0:1788572547.388982:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360529536:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000004:00080000:0.0:1788572547.391850:0:6873:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 895460842277ns, next at 896460839903ns, rc = 0 00000100:00100000:0.0:1788572547.391874:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788572547.391895:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382385664:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788572547.391974:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572547.392046:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382385664 00000100:00100000:0.0:1788572547.392054:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382385664 00000100:00100000:0.0:1788572547.392101:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1788572547.392110:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382385664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788572547.392131:0:6684:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 3407872 free 3858759680 avail 3855613952 00000100:00100000:0.0:1788572547.392175:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382385664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.392182:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1788572547.392553:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382385664:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788572547.396965:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.397125:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529664 00000100:00100000:1.0:1788572547.397136:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529664 00000100:00100000:1.0:1788572547.397200:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1300 00000100:00100000:1.0:1788572547.397211:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360529664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.400738:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35945:x1875453360528128:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 73405us (73778us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.400753:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4633 00000001:00080000:2.0:1788572547.402046:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970753 is committed 00000020:00080000:2.0:1788572547.402070:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627fb17dc00 refcount 2 committed 0 result 0 batchid 136 00000020:00080000:2.0:1788572547.402075:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c687cc00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627fb17d840 00000020:00080000:2.0:1788572547.402078:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c687c200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9628020e2800 00000020:00080000:2.0:1788572547.402105:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c97cb000 refcount 2 committed 0 result 0 batchid 138 00000020:00080000:2.0:1788572547.402109:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ea8f00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c97cb600 00000020:00080000:2.0:1788572547.402112:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ea8700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffd0dc00 00000020:00080000:2.0:1788572547.402139:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c9b79a80 refcount 2 committed 0 result 0 batchid 137 00000020:00080000:2.0:1788572547.402142:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8ade00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c9b799c0 00000020:00080000:2.0:1788572547.402144:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8adc00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffd0c400 00000020:00080000:2.0:1788572547.402158:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 135 batchid updated 00000020:00080000:2.0:1788572547.402201:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 138 committed batchid 135 00000020:00080000:2.0:1788572547.402409:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 135 00000100:00100000:0.0:1788572547.409065:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.409100:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360529792 00000100:00100000:0.0:1788572547.409110:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360529792 00000100:00100000:0.0:1788572547.409168:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1301 00000100:00100000:0.0:1788572547.409180:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360529792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.440068:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360529792:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 30875us (30998us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.440085:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1301 00000100:00100000:1.0:1788572547.440245:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2348:x1875453360529664:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 43033us (43288us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.440254:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1300 00000001:00080000:3.0:1788572547.442944:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967653 is committed 00000001:00080000:3.0:1788572547.442981:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967649 is committed 00000100:00100000:1.0:1788572547.444488:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.444549:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.444694:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360530048 00000100:00100000:3.0:1788572547.444705:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360530048 00000100:00100000:3.0:1788572547.444767:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360530304 00000100:00100000:3.0:1788572547.444771:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360530304 00000100:00100000:3.0:1788572547.444795:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4637 00000100:00100000:3.0:1788572547.444805:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35944:x1875453360530048:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.444880:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.444918:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb16432e lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0418 expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.444941:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572547.444947:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572547.444980:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.444993:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.445043:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572547.445251:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eec02a05 reply state 000000008d1833f9 transno 4294970754 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.445262:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572547.445357:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35944:x1875453360530048:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 553us (881us total) trans 4294970754 rc 0/0 00000100:00100000:3.0:1788572547.445367:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4637 00000100:00100000:3.0:1788572547.445386:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4638 00000100:00100000:3.0:1788572547.445395:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35945:x1875453360530304:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.445414:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.445426:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1642e8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03ee expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.445447:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:3.0:1788572547.445451:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572547.445466:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.445474:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.445481:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572547.445548:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b029c1e5 reply state 0000000017da4217 transno 4294970755 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.445555:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572547.445591:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35945:x1875453360530304:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 198us (1041us total) trans 4294970755 rc 0/0 00000100:00100000:3.0:1788572547.445598:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4638 00000100:00100000:3.0:1788572547.450461:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572547.450564:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.450579:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.450587:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.450593:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.450598:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.450607:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.450625:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:1.0:1788572547.450628:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.450633:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.450639:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164351 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.450672:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360530432 00000100:00100000:1.0:1788572547.450682:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360530432 00000100:00100000:1.0:1788572547.450830:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1302 00000100:00100000:1.0:1788572547.450841:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360530432:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572547.451142:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x57:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572547.451146:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x57:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:1.0:1788572547.451148:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x57:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:1.0:1788572547.451150:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x57:0x0] updating lvb ctime from disk: 1788572546 -> 1788572526 00002000:00010000:1.0:1788572547.451153:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x57:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572547.451356:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360530432:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 515us (904us total) trans 4294967654 rc 0/0 00000100:00100000:1.0:1788572547.451364:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1302 00010000:00010000:2.0:1788572547.451545:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.451553:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.451560:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.451565:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.451568:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.451574:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.451586:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:2.0:1788572547.451589:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.451593:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.451599:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164358 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572547.452302:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967648 is committed 00000001:00080000:2.0:1788572547.452315:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967653 is committed 00000001:00080000:2.0:1788572547.452324:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967651 is committed 00000100:00100000:1.0:1788572547.457345:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360529536:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 68351us (68522us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.457359:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1299 00000100:00100000:2.0:1788572547.457564:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.461494:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360530560 00000100:00100000:1.0:1788572547.461510:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360530560 00000100:00100000:2.0:1788572547.461731:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1303 00000100:00100000:2.0:1788572547.461743:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360530560:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572547.461983:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:2.0:1788572547.461986:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:2.0:1788572547.461989:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:2.0:1788572547.461991:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb ctime from disk: 1788572546 -> 1788572526 00002000:00010000:2.0:1788572547.461994:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1788572547.462087:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360530560:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 346us (4522us total) trans 4294967655 rc 0/0 00000100:00100000:2.0:1788572547.462095:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1303 00000100:00100000:2.0:1788572547.462215:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.462292:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.462314:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360530816 00000100:00100000:2.0:1788572547.462318:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360530816 00000100:00100000:2.0:1788572547.462347:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531072 00000100:00100000:2.0:1788572547.462350:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531072 00000100:00100000:2.0:1788572547.462367:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4639 00000100:00100000:2.0:1788572547.462374:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35945:x1875453360530816:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.462433:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462460:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 66 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462470:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462503:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 66 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462522:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.462537:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b33800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382385792:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572547.462605:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12300 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.462626:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4640 00000100:00100000:2.0:1788572547.462634:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35946:x1875453360531072:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.462667:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.462689:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164320 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f040a expref: 66 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462704:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:2.0:1788572547.462708:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572547.462727:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462738:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.462748:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572547.462877:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006e1a1a82 reply state 0000000009bf4d41 transno 4294970756 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.462887:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572547.462952:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35946:x1875453360531072:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 319us (660us total) trans 4294970756 rc 0/0 00000100:00100000:2.0:1788572547.462959:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4640 00010000:00010000:1.0:1788572547.467387:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.467405:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.467412:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.467418:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.467423:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.467431:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.467459:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:1.0:1788572547.467462:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.467469:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.467475:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164366 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.467982:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.468080:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531200 00000100:00100000:1.0:1788572547.468090:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531200 00010000:00010000:1.0:1788572547.468126:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.468165:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1096 00000100:00100000:1.0:1788572547.468176:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360531200:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.468202:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.468208:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.468222:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 66 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.468241:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:1.0:1788572547.468246:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.468255:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.468262:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.468282:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.468332:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360531200:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 158us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.468339:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1096 00010000:00010000:0.0:1788572547.468490:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb164263 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03c4 expref: 65 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.468503:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b33800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382385792:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.468523:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468533:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468538:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468567:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468574:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468580:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468586:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.468589:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468595:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468603:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:0.0:1788572547.468606:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572547.468609:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.468614:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb16435f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.468654:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35945:x1875453360530816:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 6280us (6440us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572547.468662:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4639 00000100:00100000:2.0:1788572547.472059:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.472100:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.472169:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531584 00000100:00100000:0.0:1788572547.472179:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531584 00000100:00100000:0.0:1788572547.472252:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4641 00000100:00100000:0.0:1788572547.472262:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35944:x1875453360531584:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.472323:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.472337:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.472346:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.472382:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.472396:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 64 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.472410:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382385920:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.472503:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca85e840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.473260:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531328 00000100:00100000:1.0:1788572547.473266:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531328 00000100:00100000:1.0:1788572547.473296:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1304 00000100:00100000:1.0:1788572547.473302:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360531328:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572547.473544:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572547.473547:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:1.0:1788572547.473550:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:1.0:1788572547.473553:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb ctime from disk: 1788572546 -> 1788572526 00002000:00010000:1.0:1788572547.473555:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572547.473640:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360531328:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 336us (1588us total) trans 4294967654 rc 0/0 00000100:00100000:1.0:1788572547.473648:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1304 00000100:00100000:2.0:1788572547.477240:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.477365:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531712 00000100:00100000:1.0:1788572547.477377:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531712 00000100:00100000:1.0:1788572547.477448:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1305 00000100:00100000:1.0:1788572547.477458:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7661050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360531712:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.477510:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.477532:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.477542:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.477548:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.477552:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.477590:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.477603:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 13 pid: 6679 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.477617:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7874380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382386048:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.477804:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.482230:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382385920:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572547.482277:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.483529:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7874380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382386048:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572547.483555:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.486160:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.486209:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.486249:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.486377:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531968 00000100:00100000:0.0:1788572547.486387:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531968 00010000:00010000:0.0:1788572547.486431:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 14 pid: 6679 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.486482:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360531840 00000100:00100000:0.0:1788572547.486487:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360531840 00010000:00010000:0.0:1788572547.486503:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 65 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.486529:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1097 00000100:00100000:0.0:1788572547.486539:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9db450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360531968:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.486556:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.486580:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 14 pid: 6679 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486593:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 14 pid: 6679 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486623:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007785cc1b/0x85ace67feb164312 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0411 expref: 14 pid: 6679 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486640:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.486645:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.486651:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486661:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486691:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.486746:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9db450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360531968:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 208us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.486754:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1097 00000100:00100000:0.0:1788572547.486802:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1098 00000100:00100000:0.0:1788572547.486810:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360531840:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.486820:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.486826:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 65 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486837:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 65 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486851:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb1642f6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03f5 expref: 65 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486866:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572547.486872:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486881:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486888:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.486922:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.486943:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360531840:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 134us (733us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.486951:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1098 00010000:00010000:0.0:1788572547.488537:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488556:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488594:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488604:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488613:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488622:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572547.488627:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488636:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488657:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.488667:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8c9164c/0x85ace67feb164374 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.488792:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7661050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360531712:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 11329us (11555us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.488801:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1305 00010000:00010000:0.0:1788572547.491473:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491493:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491529:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491535:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491545:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491551:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.491555:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491563:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491580:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:0.0:1788572547.491584:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572547.491589:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491595:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd600/0x85ace67feb16436d lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.491707:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35944:x1875453360531584:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 19441us (19600us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572547.491717:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4641 00000100:00100000:0.0:1788572547.491748:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360532224 00000100:00100000:0.0:1788572547.491756:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360532224 00000100:00100000:0.0:1788572547.491807:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4642 00000100:00100000:0.0:1788572547.491816:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35946:x1875453360532224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.491890:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491906:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491914:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491948:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.491963:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 63 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.491979:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382386176:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.492065:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.496966:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.497035:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.497081:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.497110:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360532736 00000100:00100000:0.0:1788572547.497118:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360532736 00010000:00010000:0.0:1788572547.497159:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.497522:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1099 00000100:00100000:0.0:1788572547.497534:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360532736:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.497550:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.497557:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497571:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497596:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:0.0:1788572547.497601:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497612:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497620:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497640:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.497689:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360532736:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 158us (607us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.497698:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1099 00010000:00010000:0.0:1788572547.497753:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb16429b lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03cb expref: 63 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.497770:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382386176:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.497789:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497800:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497806:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497837:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.497897:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.498132:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360532608 00000100:00100000:2.0:1788572547.498139:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360532608 00000100:00100000:2.0:1788572547.498171:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1306 00000100:00100000:2.0:1788572547.498179:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360532608:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.498230:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.498246:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.498254:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572547.498258:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572547.498262:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.498291:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 12 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.498304:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 12 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.498322:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382386304:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572547.498377:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00010000:00010000:0.0:1788572547.498508:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.498517:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.498521:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.498529:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.498539:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:0.0:1788572547.498542:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572547.498546:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.498551:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb16437b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.498593:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35946:x1875453360532224:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 6778us (12343us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572547.498601:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4642 00000100:00100000:0.0:1788572547.498616:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360532480 00000100:00100000:0.0:1788572547.498619:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360532480 00000100:00100000:0.0:1788572547.498641:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4643 00000100:00100000:0.0:1788572547.498647:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35945:x1875453360532480:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.498702:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.498713:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.498722:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.500355:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.500378:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.500386:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.500420:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.500431:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 63 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.500456:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382386432:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.500530:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.505322:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.505420:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382386304:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.505462:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.505551:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382386432:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.505571:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.505642:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360532864 00000100:00100000:1.0:1788572547.505650:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360532864 00000100:00100000:1.0:1788572547.505715:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1307 00000100:00100000:1.0:1788572547.505726:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360532864:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.505769:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.505794:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.505802:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.505807:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.505813:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.505851:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.505865:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 11 pid: 10416 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.505881:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7874380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382386560:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.505979:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572547.508307:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7874380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382386560:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572547.508337:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.510812:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.510905:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.511695:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360532992 00000100:00100000:3.0:1788572547.511706:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360532992 00000100:00100000:1.0:1788572547.512797:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.512949:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360533248 00000100:00100000:3.0:1788572547.512956:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360533248 00010000:00010000:3.0:1788572547.512998:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.513062:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360533120 00000100:00100000:3.0:1788572547.513065:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360533120 00010000:00010000:3.0:1788572547.513081:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 12 pid: 10416 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.513108:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1100 00000100:00100000:3.0:1788572547.513117:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360533248:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.513133:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.513140:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513153:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513178:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1642a2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03d9 expref: 64 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513214:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:3.0:1788572547.513219:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513229:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513237:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513262:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.513310:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2350:x1875453360533248:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 194us (2404us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.513318:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1100 00000100:00100000:3.0:1788572547.513334:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1101 00000100:00100000:3.0:1788572547.513339:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360533120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.513347:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.513367:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 12 pid: 10416 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513377:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 12 pid: 10416 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513391:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094303b96/0x85ace67feb164343 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x57:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f042d expref: 12 pid: 10416 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513410:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572547.513413:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572547.513418:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513427:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513460:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.513480:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360533120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 141us (692us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.513487:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1101 00010000:00010000:3.0:1788572547.513512:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513521:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513546:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513554:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513562:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513570:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572547.513575:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513584:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513596:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513603:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001da6caee/0x85ace67feb164397 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.513662:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360532864:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 7936us (8348us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.513672:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1307 00010000:00010000:3.0:1788572547.513694:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.513702:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572547.514786:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970757 00000001:00080000:3.0:1788572547.515671:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970758 00000001:00080000:3.0:1788572547.515731:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970759 00000100:00100000:3.0:1788572547.515830:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382386688:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572547.516346:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572547.516429:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9faf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.518049:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.518200:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360533504 00000100:00100000:3.0:1788572547.518209:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360533504 00000100:00100000:3.0:1788572547.518260:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4644 00000100:00100000:3.0:1788572547.518270:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35946:x1875453360533504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.518352:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518370:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518382:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518614:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518627:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518635:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518669:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.518682:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 67 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.518701:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382386816:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572547.518776:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d300 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.522090:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382386816:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.522122:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.522247:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.522299:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1102 00000100:00100000:2.0:1788572547.522310:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360532992:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572547.522325:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.522348:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.522362:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.522385:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb16434a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0434 expref: 13 pid: 20911 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.522406:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572547.522411:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572547.522417:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.522427:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.522461:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572547.522516:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360532992:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 208us (11711us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.522523:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1102 00000100:00100000:1.0:1788572547.523338:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572547.523422:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523443:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523479:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523488:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523496:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523503:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572547.523508:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523517:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523531:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.523539:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074d56732/0x85ace67feb164382 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.523602:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360532608:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 25423us (26566us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.523614:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1306 00000100:00100000:3.0:1788572547.523713:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360533632 00000100:00100000:3.0:1788572547.523721:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360533632 00010000:00010000:3.0:1788572547.523747:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 68 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.523778:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1103 00000100:00100000:3.0:1788572547.523786:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360533632:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.523797:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.523804:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 68 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523815:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 68 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523830:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb1642da lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f03e7 expref: 68 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523844:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572547.523848:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523859:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523867:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523889:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.523921:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2349:x1875453360533632:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 136us (586us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.523927:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1103 00010000:00010000:3.0:1788572547.523946:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.523952:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572547.524161:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970761 00000001:00080000:3.0:1788572547.524317:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970762 00000001:00080000:3.0:1788572547.524365:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970763 00000100:00100000:3.0:1788572547.524487:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382386944:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572547.524532:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572547.527715:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.527849:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382386688 00000100:00100000:0.0:1788572547.527860:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382386688 00000100:00100000:0.0:1788572547.527926:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382386944 00000100:00100000:0.0:1788572547.527929:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382386944 00000100:00100000:0.0:1788572547.527950:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1788572547.527958:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382386688:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1788572547.528103:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382386688:12345-0@lo:1000:mdt00_004.0 Request processed in 145us (11758us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.528111:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1788572547.528130:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1788572547.528137:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382386944:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572547.528408:0:23058:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967651 gen 2 00000004:00080000:3.0:1788572547.528414:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967650 committed transno 4294967651 00000004:00080000:3.0:1788572547.528442:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967651 committed transno 4294967651 00000100:00100000:3.0:1788572547.528461:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382386944:0@lo:1000:mdt00_005.0 00000001:00080000:3.0:1788572547.528880:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970764 00000100:00100000:3.0:1788572547.528980:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382387072:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572547.529067:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572547.529095:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.529124:0:22536:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967649 gen 2 00000004:00080000:3.0:1788572547.529128:0:22536:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967649 committed transno 4294967649 00000100:00100000:3.0:1788572547.529143:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382386688:0@lo:1000:mdt00_004.0 00000001:00080000:3.0:1788572547.529427:0:22536:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970760 00000100:00100000:3.0:1788572547.529482:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382387200:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1788572547.529511:0:22536:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572547.529522:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.529600:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382387072 00000100:00100000:1.0:1788572547.529607:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382387072 00000100:00100000:1.0:1788572547.529636:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382387200 00000100:00100000:1.0:1788572547.529639:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382387200 00000100:00100000:1.0:1788572547.529654:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1788572547.529661:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382387072:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1788572547.529862:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:23058:x1875453382387072:12345-0@lo:1000:mdt00_005.0 Request processed in 201us (794us total) trans 4294967655 rc 0/0 00000100:00100000:1.0:1788572547.529872:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1788572547.529891:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:1.0:1788572547.529897:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:22536:x1875453382387200:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1788572547.530058:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:22536:x1875453382387200:12345-0@lo:1000:mdt00_004.0 Request processed in 160us (545us total) trans 4294967656 rc 0/0 00000100:00100000:1.0:1788572547.530064:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:1.0:1788572547.530100:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:22536:1875453382387200:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1788572547.530142:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530152:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530159:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530165:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.530170:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530180:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530212:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9627fb1e7d00) 00010000:00010000:1.0:1788572547.530216:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572547.530221:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530227:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164390 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530260:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530267:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530273:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530278:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6a6800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.530281:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530286:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530294:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627c2abcb00) 00010000:00010000:1.0:1788572547.530297:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572547.530300:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530308:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb15feaf lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530331:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530345:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530355:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530363:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530369:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530375:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530380:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.530383:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530389:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530396:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627c2abcb00) 00010000:00010000:1.0:1788572547.530399:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.530402:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530409:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb1643ac lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530419:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530425:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530430:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530447:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.530450:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530456:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530464:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627c2abcb00) 00010000:00010000:1.0:1788572547.530466:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572547.530469:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.530475:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb164389 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.530692:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35945:x1875453360532480:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 32041us (33733us total) trans 4294970766 rc 0/0 00000100:00100000:1.0:1788572547.530703:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4643 00000020:00080000:1.0:1788572547.530709:0:22536:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1788572547.531543:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c49880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382387072:0@lo:1000:mdt00_005.0 00010000:00010000:3.0:1788572547.531585:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531594:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531600:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531605:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.531609:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531617:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531629:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9627e7a95300) 00010000:00010000:3.0:1788572547.531632:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572547.531636:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531642:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643a5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531671:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531678:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531683:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531688:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.531690:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531695:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531703:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:3.0:1788572547.531705:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572547.531712:0:23058:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 2/0,0 mode: --/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 174 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531728:0:23058:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 174 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531738:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 174 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.531789:0:23058:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fe00/0x85ace67feb1639db lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f017f expref: 174 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.531806:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382387328:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572547.531894:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.532032:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382386944:12345-0@lo:1000:mdt00_005.0 Request processed in 3893us (7496us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.532043:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:1.0:1788572547.535659:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.535822:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360533888 00000100:00100000:3.0:1788572547.535832:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360533888 00000100:00100000:3.0:1788572547.535893:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 965 00000100:00100000:3.0:1788572547.535904:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35945:x1875453360533888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572547.536819:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x9f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572547.536940:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970767 00010000:00010000:3.0:1788572547.536994:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.537530:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.537544:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.537553:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.537559:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.537565:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.537624:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572547.537743:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35945:x1875453360533888:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1837us (2095us total) trans 4294970768 rc 0/0 00000100:00100000:3.0:1788572547.537754:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 965 00000020:00080000:3.0:1788572547.537760:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788572547.540891:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.540986:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360534144 00000100:00100000:1.0:1788572547.540996:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360534144 00000100:00100000:1.0:1788572547.541069:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4645 00000100:00100000:1.0:1788572547.541078:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35944:x1875453360534144:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.541158:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 3/0,1 mode: --/PW res: [0x200000401:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541174:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541186:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541419:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541430:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541451:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541485:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.541499:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.541513:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7875c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382387456:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572547.541583:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.541845:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382387328:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572547.541870:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.541879:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb15ff03 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.541904:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.541914:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 174 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.541923:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.541945:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 174 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.541953:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 174 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.541962:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb649500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382387584:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.541992:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.545734:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.545888:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb649500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382387584:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.545922:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.545986:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7875c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382387456:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.545999:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.546086:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360534272 00000100:00100000:1.0:1788572547.546092:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360534272 00000100:00100000:1.0:1788572547.546145:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 966 00000100:00100000:1.0:1788572547.546153:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd1450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360534272:12345-192.168.202.40@tcp:49:diff.0 00000100:00100000:1.0:1788572547.546281:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd1450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360534272:12345-192.168.202.40@tcp:49:diff.0 Request processed in 128us (559us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.546289:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 966 00010000:00010000:2.0:1788572547.546735:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572547.546743:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.546750:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572547.546755:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.546764:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.546781:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9f:0x0].0x0 (ffff9626c1d8fe00) 00010000:00010000:2.0:1788572547.546784:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572547.546789:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.546800:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb1643b3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x9f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.546821:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.546823:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.546824:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572547.548134:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572547.549225:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.549352:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360534528 00000100:00100000:0.0:1788572547.549362:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360534528 00010000:00010000:0.0:1788572547.549403:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 175 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.549452:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1104 00000100:00100000:0.0:1788572547.549462:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9d9050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:2349:x1875453360534528:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.549478:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.549484:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 175 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.549496:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 175 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.549526:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16396b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f015c expref: 175 pid: 22536 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.549542:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:0.0:1788572547.549547:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.549558:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.549566:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.549594:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.549647:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9d9050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:2349:x1875453360534528:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 186us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.549655:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1104 00010000:00010000:3.0:1788572547.550386:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550397:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550410:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550417:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550423:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550429:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.550444:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550453:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550465:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:3.0:1788572547.550468:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.550472:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550478:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1643c8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550491:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550496:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550502:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550508:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.550510:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550516:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550524:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xac:0x0].0x0 (ffff9627ffc16600) 00010000:00010000:3.0:1788572547.550527:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572547.550529:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.550538:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16439e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xac:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.550717:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35946:x1875453360533504:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 32447us (32674us total) trans 4294970765 rc 0/0 00000100:00100000:3.0:1788572547.550727:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4644 00000020:00080000:3.0:1788572547.550731:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:1.0:1788572547.553182:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.553321:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360534400 00000100:00100000:3.0:1788572547.553333:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360534400 00010000:00010000:3.0:1788572547.553375:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.553421:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1105 00000100:00100000:3.0:1788572547.553431:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360534400:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.553460:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.553468:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.553484:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.553510:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb16430b lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0403 expref: 74 pid: 23058 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.553528:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:3.0:1788572547.553534:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.553545:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.553553:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.553584:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.553653:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2349:x1875453360534400:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 222us (479us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.553662:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1105 00010000:00010000:1.0:1788572547.554638:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.554653:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572547.554888:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970769 00000100:00100000:1.0:1788572547.557412:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.557511:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360534784 00000100:00100000:3.0:1788572547.557524:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360534784 00000100:00100000:3.0:1788572547.557585:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 967 00000100:00100000:3.0:1788572547.557596:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35946:x1875453360534784:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572547.557987:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa0:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572547.558097:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970771 00010000:00010000:3.0:1788572547.558147:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.558165:0:14530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.558177:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.558185:0:14530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.558190:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.558197:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.558218:0:14530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572547.558325:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35946:x1875453360534784:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 728us (919us total) trans 4294970772 rc 0/0 00000100:00100000:3.0:1788572547.558334:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 967 00000020:00080000:3.0:1788572547.558340:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572547.558457:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572547.558465:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.558472:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572547.558478:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.558488:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.558503:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa0:0x0].0x0 (ffff9626c1d8f700) 00010000:00010000:2.0:1788572547.558506:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572547.558512:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.558519:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1643cf lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.558538:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.558541:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572547.558543:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1788572547.560384:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970770 00000001:00080000:0.0:1788572547.560500:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970773 00000100:00100000:0.0:1788572547.560604:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382387712:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572547.560671:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572547.560697:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582af00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.560765:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382387712 00000100:00100000:1.0:1788572547.560773:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382387712 00000100:00100000:1.0:1788572547.560826:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1788572547.560836:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382387712:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572547.560945:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382387712:12345-0@lo:1000:mdt00_003.0 Request processed in 110us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.560953:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1788572547.561158:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382387712:0@lo:1000:mdt00_003.0 00000001:00080000:2.0:1788572547.561549:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970774 00000100:00100000:2.0:1788572547.561612:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b31180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382387840:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1788572547.561648:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.561683:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582af00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.561813:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382387840 00000100:00100000:3.0:1788572547.561818:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382387840 00000100:00100000:3.0:1788572547.561862:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:3.0:1788572547.561870:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:9040:x1875453382387840:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572547.562081:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:9040:x1875453382387840:12345-0@lo:1000:mdt00_003.0 Request processed in 211us (432us total) trans 4294967657 rc 0/0 00000100:00100000:3.0:1788572547.562088:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1788572547.563504:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b31180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382387840:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1788572547.563551:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563561:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563569:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563575:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.563580:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563588:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563603:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.563606:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.563611:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563617:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1643c1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563649:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563655:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563661:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563666:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.563668:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563675:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563682:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9627c2abc000) 00010000:00010000:2.0:1788572547.563685:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572547.563688:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563693:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb15fe85 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xab:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563717:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563731:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563739:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563746:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563750:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563756:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563761:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.563764:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563769:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563775:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9627c2abc000) 00010000:00010000:2.0:1788572547.563777:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.563780:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563785:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643d6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xab:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563793:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563798:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563803:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563808:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.563811:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563821:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563827:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9627c2abc000) 00010000:00010000:2.0:1788572547.563830:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572547.563833:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 2/0,0 mode: --/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.563838:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb1643ba lrc: 0/0,0 mode: --/PW res: [0x200000401:0xab:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.564115:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35944:x1875453360534144:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 23033us (23227us total) trans 4294970775 rc 0/0 00000100:00100000:2.0:1788572547.564128:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4645 00000020:00080000:2.0:1788572547.564134:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1788572547.564722:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572547.564729:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572547.564777:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 166 count 101 00000040:00080000:3.0:1788572547.564785:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:3.0:1788572547.565092:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382387968:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572547.565166:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.568078:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.568490:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360534912 00000100:00100000:0.0:1788572547.568502:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360534912 00000100:00100000:0.0:1788572547.568578:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1308 00000100:00100000:0.0:1788572547.568591:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:35867:x1875453360534912:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572547.568609:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.568633:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007c9f445a/0x85ace67feb1643dd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f043b expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572547.568660:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x57:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:0.0:1788572547.568683:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=895s v=(0 0) 00010000:00010000:0.0:1788572547.568688:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.568697:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007c9f445a/0x85ace67feb1643dd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f043b expref: 53 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.568715:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007c9f445a/0x85ace67feb1643dd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f043b expref: 53 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.568727:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd600, rc 0) 00000100:00100000:0.0:1788572547.568791:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:35867:x1875453360534912:12345-192.168.202.40@tcp:101:diff.0 Request processed in 203us (724us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.568798:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1308 00000100:00080000:3.0:1788572547.572123:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572547.573478:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.573555:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382387968:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572547.573882:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535168 00000100:00100000:0.0:1788572547.573890:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535168 00000100:00100000:0.0:1788572547.573953:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 968 00000100:00100000:0.0:1788572547.573964:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35944:x1875453360535168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.573997:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.574041:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.574063:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.574082:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535296 00000100:00100000:1.0:1788572547.574085:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535296 00000100:00100000:1.0:1788572547.574115:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535424 00000100:00100000:1.0:1788572547.574118:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535424 00000100:00100000:1.0:1788572547.574132:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535552 00000100:00100000:1.0:1788572547.574135:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535552 00000100:00100000:1.0:1788572547.574152:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4646 00000100:00100000:1.0:1788572547.574158:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35949:x1875453360535296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572547.574176:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa9:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572547.574281:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35949:x1875453360535296:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 122us (283us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.574289:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4646 00000100:00100000:1.0:1788572547.574305:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4647 00000100:00100000:1.0:1788572547.574311:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35948:x1875453360535424:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572547.574324:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa4:0x0] from 18446744073709551615 #0 00000004:00080000:0.0:1788572547.574333:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa1:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000100:00100000:1.0:1788572547.574363:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35948:x1875453360535424:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 52us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.574369:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4647 00000001:00080000:0.0:1788572547.574376:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970776 00000100:00100000:1.0:1788572547.574381:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4648 00000100:00100000:1.0:1788572547.574387:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35947:x1875453360535552:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572547.574399:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xaa:0x0] from 18446744073709551615 #0 00010000:00010000:0.0:1788572547.574418:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.574450:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.574450:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35947:x1875453360535552:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 62us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.574457:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4648 00010000:00010000:0.0:1788572547.574468:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.574476:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.574482:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.574488:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.574502:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572547.574554:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35944:x1875453360535168:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 592us (1085us total) trans 4294970777 rc 0/0 00000100:00100000:0.0:1788572547.574562:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 968 00000020:00080000:0.0:1788572547.574687:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572547.574790:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572547.574798:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.574810:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572547.574814:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.574822:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.574835:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa1:0x0].0x0 (ffff9627fb1e7a00) 00010000:00010000:3.0:1788572547.574838:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572547.574842:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.574849:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1643e4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.574864:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572547.574867:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572547.574869:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572547.580365:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.580594:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535808 00000100:00100000:2.0:1788572547.580605:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535808 00000100:00100000:2.0:1788572547.580676:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4649 00000100:00100000:2.0:1788572547.580687:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35947:x1875453360535808:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.580775:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580795:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580808:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580828:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580837:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580844:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580885:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580892:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580900:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572547.580951:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaa:0x0]: returning size 3 00000004:00200000:2.0:1788572547.580956:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572547.580969:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580980:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580986:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.580994:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581000:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.581028:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581037:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581049:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572547.581054:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.581058:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581065:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1643eb lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581078:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581084:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581090:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581094:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.581097:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581103:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581110:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3ca8efa5 (ffff9627e7a95900) 00010000:00010000:2.0:1788572547.581113:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.581116:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581125:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1643f2 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3ca8efa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581138:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581143:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581148:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581153:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.581155:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581161:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581167:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:2.0:1788572547.581169:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572547.581172:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.581177:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1643f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.581597:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.581678:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535680 00000100:00100000:0.0:1788572547.581684:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535680 00002000:00100000:0.0:1788572547.581724:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x57:0x0] (0->4095) req@ffff9627e7877480 x1875453360535680/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:208/0 lens 488/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572547.581734:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7877480 with x1875453360535680 ext(0->4095) 00002000:00010000:0.0:1788572547.581738:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7877480 00000100:00100000:0.0:1788572547.581770:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 291 00000100:00100000:0.0:1788572547.581779:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:2348:x1875453360535680:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572547.581994:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:2348:x1875453360535680:12345-192.168.202.40@tcp:3:diff.0 Request processed in 215us (398us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572547.582029:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 291 00002000:00100000:0.0:1788572547.582037:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x57:0x0] (0->4095) req@ffff9627e7877480 x1875453360535680/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:208/0 lens 488/4536 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572547.582045:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7877480 with x1875453360535680 ext(0->4095) 00002000:00010000:0.0:1788572547.582048:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7877480 00000100:00100000:3.0:1788572547.582418:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35947:x1875453360535808:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1731us (2061us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.582428:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4649 00000100:00100000:1.0:1788572547.584717:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.584804:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360535936 00000100:00100000:2.0:1788572547.584814:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360535936 00000100:00100000:2.0:1788572547.584881:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4650 00000100:00100000:2.0:1788572547.584892:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35948:x1875453360535936:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.584984:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.585041:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.585054:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585096:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585106:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585114:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585147:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585154:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585160:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.585199:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa4:0x0]: returning size 3 00000004:00200000:3.0:1788572547.585203:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.585216:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585227:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585233:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585241:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585247:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.585252:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585260:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585271:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572547.585275:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.585279:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585285:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164400 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585296:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585301:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585308:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585312:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.585315:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585320:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585327:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x24857360 (ffff9627e7a95f00) 00010000:00010000:3.0:1788572547.585330:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.585332:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585338:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164407 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x24857360 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585352:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585358:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585363:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585368:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.585371:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585376:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585396:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9627fb1e7400) 00010000:00010000:3.0:1788572547.585399:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.585402:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.585408:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16440e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.585662:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35948:x1875453360535936:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 769us (956us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.585670:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4650 00000100:00100000:0.0:1788572547.590245:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.590353:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360536064 00000100:00100000:1.0:1788572547.590363:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360536064 00000100:00100000:1.0:1788572547.590428:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4651 00000100:00100000:1.0:1788572547.590449:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35949:x1875453360536064:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.590531:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590550:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590562:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590577:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590584:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590590:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590621:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590629:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590636:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.590682:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa9:0x0]: returning size 3 00000004:00200000:1.0:1788572547.590687:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572547.590706:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590721:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590730:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590737:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590743:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.590747:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590756:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590768:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572547.590771:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.590777:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590783:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164415 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590798:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590809:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590815:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590818:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.590821:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590824:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590829:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x66e6b519 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.590831:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.590832:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590836:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16441c lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x66e6b519 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590846:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590850:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590854:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590857:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.590859:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590863:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590867:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9627ffc16a00) 00010000:00010000:1.0:1788572547.590868:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.590870:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.590873:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb164423 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.590964:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35949:x1875453360536064:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 526us (726us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.590971:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4651 00000100:00100000:2.0:1788572547.597236:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.597284:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.597391:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360536576 00000100:00100000:1.0:1788572547.597400:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360536576 00000100:00100000:2.0:1788572547.597408:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360536320 00000100:00100000:2.0:1788572547.597411:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360536320 00000100:00100000:1.0:1788572547.597468:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4652 00000100:00100000:2.0:1788572547.597468:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 969 00000100:00100000:1.0:1788572547.597479:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35948:x1875453360536576:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.597479:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+174:35867:x1875453360536320:12345-192.168.202.40@tcp:35:diff.0 00010000:00010000:1.0:1788572547.597503:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.597529:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16442a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0442 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597624:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597640:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597656:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.597667:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa4:0x0]: returning size 3 00010000:00010000:1.0:1788572547.597735:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597742:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597749:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597755:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.597759:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597767:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597780:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9627fb1e7400) 00010000:00010000:1.0:1788572547.597783:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572547.597788:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597794:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164431 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.597824:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16442a lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0442 expref: 82 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.597838:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360536320:12345-192.168.202.40@tcp:35:diff.0 Request processed in 347us (541us total) trans 4294970779 rc 0/0 00010000:00010000:1.0:1788572547.597840:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572547.597842:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572547.597847:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97200/0x85ace67feb16442a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0442 expref: 82 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.597847:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 969 00010000:00010000:1.0:1788572547.597855:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572547.597877:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35948:x1875453360536576:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 401us (653us total) trans 4294970778 rc 301/301 00000100:00100000:1.0:1788572547.597884:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4652 00000100:00100000:2.0:1788572547.599905:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.601275:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360536832 00000100:00100000:3.0:1788572547.601290:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360536832 00000100:00100000:3.0:1788572547.601349:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4653 00000100:00100000:3.0:1788572547.601360:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35947:x1875453360536832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.601382:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.601405:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16443f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0449 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601526:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601541:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601553:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.601565:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaa:0x0]: returning size 3 00010000:00010000:3.0:1788572547.601666:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601675:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601682:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601687:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.601691:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601699:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601713:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:3.0:1788572547.601716:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.601721:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601728:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164446 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601777:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16443f lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0449 expref: 83 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601787:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572547.601790:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572547.601794:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16443f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0449 expref: 83 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.601803:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc800, rc 0) 00000100:00100000:3.0:1788572547.601889:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35947:x1875453360536832:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 530us (1993us total) trans 4294970780 rc 301/301 00000100:00100000:3.0:1788572547.601897:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4653 00000100:00100000:0.0:1788572547.603065:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.603134:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360536960 00000100:00100000:1.0:1788572547.603143:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360536960 00000100:00100000:1.0:1788572547.603185:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4654 00000100:00100000:1.0:1788572547.603193:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+175:35867:x1875453360536960:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572547.603213:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.603229:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164454 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0450 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603286:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603300:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603310:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603325:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603332:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603337:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603700:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb164469 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603709:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb164469 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603716:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb164469 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.603751:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa3:0x0]: returning size 3 00000004:00200000:1.0:1788572547.603756:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572547.603778:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb164469 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603789:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb164469 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603797:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603803:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603809:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603819:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.603824:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603833:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603844:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572547.603847:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.603852:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603857:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb16445b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603870:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603876:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603882:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603887:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.603890:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603897:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603909:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x6e9b5ca3 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.603911:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.603914:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603919:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164462 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603954:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164454 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0450 expref: 177 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603967:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96400/0x85ace67feb164469 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0450 expref: 176 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.603974:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627c8ed8600) 00010000:00010000:1.0:1788572547.603977:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572547.603979:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96400, rc 0) 00000100:00100000:1.0:1788572547.617471:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:35867:x1875453360536960:12345-192.168.202.40@tcp:101:diff.0 Request processed in 14263us (14395us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.617487:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4654 00000100:00100000:0.0:1788572547.623180:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.623367:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.623410:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.623453:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.624080:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360537472 00000100:00100000:2.0:1788572547.624092:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360537472 00000100:00100000:2.0:1788572547.624148:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360537088 00000100:00100000:2.0:1788572547.624152:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360537088 00000100:00100000:2.0:1788572547.624169:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360537216 00000100:00100000:2.0:1788572547.624172:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360537216 00000100:00100000:1.0:1788572547.624300:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360537600 00000100:00100000:1.0:1788572547.624305:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360537600 00000100:00100000:1.0:1788572547.624350:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1309 00000100:00100000:1.0:1788572547.624361:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+41:2350:x1875453360537600:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572547.624396:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.624419:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb164470 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f045e expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.624456:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4655 00000100:00100000:2.0:1788572547.624466:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35949:x1875453360537472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.624483:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.624499:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164477 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0457 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.624535:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4656 00000100:00100000:0.0:1788572547.624544:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35948:x1875453360537088:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572547.624562:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa4:0x0] from 18446744073709551615 #0 00010000:00010000:2.0:1788572547.624615:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624630:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624642:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572547.624653:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa9:0x0]: returning size 3 00000100:00100000:0.0:1788572547.624678:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35948:x1875453360537088:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 135us (1310us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.624685:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4656 00000100:00100000:0.0:1788572547.624706:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4657 00000100:00100000:0.0:1788572547.624712:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35947:x1875453360537216:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572547.624724:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xaa:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572547.624763:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35947:x1875453360537216:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 51us (1353us total) trans 0 rc 0/0 00010000:00010000:2.0:1788572547.624766:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.624770:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4657 00010000:00010000:2.0:1788572547.624773:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624780:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624786:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572547.624790:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624797:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624809:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9627ffc16a00) 00010000:00010000:2.0:1788572547.624813:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572547.624817:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.624823:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb16447e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.625067:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164477 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0457 expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.625079:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572547.625081:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572547.625085:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164477 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0457 expref: 84 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.625093:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00002000:00010000:1.0:1788572547.625309:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x56:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:1.0:1788572547.625326:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=895s v=(0 0) 00010000:00010000:1.0:1788572547.625331:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.625341:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb164470 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f045e expref: 42 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.625356:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f0b7d0ec/0x85ace67feb164470 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f045e expref: 42 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.625366:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97200, rc 0) 00000100:00100000:1.0:1788572547.625392:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2350:x1875453360537600:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1034us (1939us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.625398:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1309 00000100:00080000:1.0:1788572547.625424:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572547.630366:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.630642:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360537728 00000100:00100000:0.0:1788572547.630653:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360537728 00000100:00100000:0.0:1788572547.630750:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4658 00000100:00100000:0.0:1788572547.630762:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35947:x1875453360537728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.630786:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.630809:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb16448c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0465 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.630893:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.630910:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.630923:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.630938:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.630945:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.630950:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.631154:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35949:x1875453360537472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6688us (7984us total) trans 4294970781 rc 301/301 00000100:00100000:2.0:1788572547.631163:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4655 00010000:00010000:0.0:1788572547.631355:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1644a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631365:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1644a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631372:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1644a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.631418:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:0.0:1788572547.631422:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572547.631454:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1644a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631466:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631472:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631479:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631484:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.631489:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631495:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631507:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572547.631510:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.631514:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631520:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb164493 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631532:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631542:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631548:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631553:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.631556:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631562:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631581:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9627fb1e7800) 00010000:00010000:0.0:1788572547.631584:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.631586:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631592:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb16449a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631624:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb16448c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0465 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631636:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1644a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0465 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.631643:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.631645:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572547.631648:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd400, rc 0) 00000100:00100000:0.0:1788572547.631682:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35947:x1875453360537728:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 922us (1329us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.631688:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4658 00000100:00100000:3.0:1788572547.633709:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.638248:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360537984 00000100:00100000:3.0:1788572547.638349:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.638451:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360537984 00000100:00100000:1.0:1788572547.638509:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360538112 00000100:00100000:1.0:1788572547.638513:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360538112 00000100:00100000:1.0:1788572547.638536:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4659 00000100:00100000:1.0:1788572547.638547:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+176:35867:x1875453360537984:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572547.638583:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.638606:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1644a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f046c expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.638727:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1644af lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.638743:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1644af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.638756:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1644af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.638768:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa3:0x0]: returning size 3 00010000:00010000:1.0:1788572547.638907:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1644a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f046c expref: 179 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.638926:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1644af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f046c expref: 178 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.638934:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627c8ed8600) 00010000:00010000:1.0:1788572547.638937:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572547.638941:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96200, rc 0) 00000100:00100000:1.0:1788572547.639024:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360537984:12345-192.168.202.40@tcp:101:diff.0 Request processed in 456us (5304us total) trans 4294970782 rc 0/0 00000100:00100000:1.0:1788572547.639033:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4659 00000100:00100000:1.0:1788572547.639053:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4660 00000100:00100000:1.0:1788572547.639060:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35949:x1875453360538112:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572547.639082:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa9:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572547.639143:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35949:x1875453360538112:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 83us (803us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.639150:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4660 00000100:00100000:3.0:1788572547.639875:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.640035:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360538368 00000100:00100000:3.0:1788572547.640043:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360538368 00000100:00100000:3.0:1788572547.640087:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4661 00000100:00100000:3.0:1788572547.640095:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35947:x1875453360538368:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.640108:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.640110:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572547.640132:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb1644a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0465 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.640153:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.640158:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572547.640168:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1644bd lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0473 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572547.640900:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x58:0x0], index 1: 0 00010000:00010000:3.0:1788572547.641131:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1644c4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.641145:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1644c4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.641155:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1644c4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.641213:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1644bd lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0473 expref: 86 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.641228:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1644c4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0473 expref: 85 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.641236:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572547.641238:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572547.641241:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572547.641298:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35947:x1875453360538368:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1204us (1427us total) trans 4294970783 rc 0/0 00000100:00100000:3.0:1788572547.641307:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4661 00000100:00100000:1.0:1788572547.642172:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.642643:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360538496 00000100:00100000:1.0:1788572547.642653:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360538496 00000100:00100000:1.0:1788572547.642738:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4662 00000100:00100000:1.0:1788572547.642748:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35948:x1875453360538496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.642764:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.642782:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1644d2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f047a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.642843:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.642856:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.642866:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.642881:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.642889:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.642894:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643211:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1644e7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643221:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1644e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643231:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1644e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.643257:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:1.0:1788572547.643261:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572547.643277:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1644e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643289:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643295:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643302:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643308:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.643312:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643320:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643331:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572547.643334:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.643339:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643350:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb1644d9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643361:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643367:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643372:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643377:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.643516:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643523:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643530:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9627c8ed8300) 00010000:00010000:1.0:1788572547.643532:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.643535:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643541:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb1644e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643585:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1644d2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f047a expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643604:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1644e7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f047a expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.643611:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572547.643613:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572547.643616:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:1.0:1788572547.643689:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35948:x1875453360538496:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 943us (1521us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.643698:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4662 00000100:00100000:1.0:1788572547.644708:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.644750:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.644777:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360538752 00000100:00100000:1.0:1788572547.644785:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360538752 00000100:00100000:1.0:1788572547.644830:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 316 00000100:00100000:1.0:1788572547.644839:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:2347:x1875453360538752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.644852:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.644869:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1644af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f046c expref: 178 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.644882:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa3:0x0].0x0 (ffff9627c8ed8600) 00010000:00010000:1.0:1788572547.644886:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.644984:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:2347:x1875453360538752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 145us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.644992:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 316 00000100:00100000:0.0:1788572547.645293:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360538624 00000100:00100000:0.0:1788572547.645299:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360538624 00002000:00100000:0.0:1788572547.645335:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x56:0x0] (0->4095) req@ffff9627f56f2300 x1875453360538624/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:208/0 lens 488/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572547.645345:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f56f2300 with x1875453360538624 ext(0->4095) 00002000:00010000:0.0:1788572547.645350:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f56f2300 00000100:00100000:0.0:1788572547.645374:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 292 00000100:00100000:0.0:1788572547.645382:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2349:x1875453360538624:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572547.645854:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2349:x1875453360538624:12345-192.168.202.40@tcp:3:diff.0 Request processed in 470us (1150us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572547.645863:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 292 00002000:00100000:0.0:1788572547.645870:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x56:0x0] (0->4095) req@ffff9627f56f2300 x1875453360538624/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:208/0 lens 488/4536 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572547.645878:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f56f2300 with x1875453360538624 ext(0->4095) 00002000:00010000:0.0:1788572547.645881:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f56f2300 00000100:00100000:2.0:1788572547.651709:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.651784:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360539008 00000100:00100000:3.0:1788572547.651797:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360539008 00000100:00100000:3.0:1788572547.651873:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4663 00000100:00100000:3.0:1788572547.651888:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35948:x1875453360539008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.651932:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.651937:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572547.651974:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1644e7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f047a expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.651995:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.652046:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572547.652063:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1644ee lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0481 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572547.652594:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x59:0x0], index 0: 0 00010000:00010000:3.0:1788572547.652794:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1644f5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.652809:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1644f5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.652818:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1644f5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.652879:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1644ee lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0481 expref: 88 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.652894:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1644f5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0481 expref: 87 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.652908:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572547.652910:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572547.652913:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc400, rc 0) 00000100:00100000:3.0:1788572547.652969:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35948:x1875453360539008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1086us (1272us total) trans 4294970784 rc 0/0 00000100:00100000:3.0:1788572547.652975:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4663 00000100:00100000:1.0:1788572547.653543:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.653581:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.653920:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360539392 00000100:00100000:2.0:1788572547.653929:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360539392 00000100:00100000:2.0:1788572547.653980:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 970 00000100:00100000:2.0:1788572547.653989:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:35867:x1875453360539392:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572547.654540:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360539136 00000100:00100000:3.0:1788572547.654545:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360539136 00000100:00100000:3.0:1788572547.654585:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4664 00000100:00100000:3.0:1788572547.654593:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35949:x1875453360539136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.654603:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.654619:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164503 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0488 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.654668:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.654682:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.654692:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.654708:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.654714:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.654721:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655026:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164518 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655036:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164518 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655043:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164518 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.655064:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572547.655068:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.655081:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164518 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655091:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655097:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655103:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655108:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.655112:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655119:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655131:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572547.655133:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.655137:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655143:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16450a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655153:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655159:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655164:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655168:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.655171:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655176:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655183:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8f400) 00010000:00010000:3.0:1788572547.655198:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.655201:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655207:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164511 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655241:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc600/0x85ace67feb164503 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0488 expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655253:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164518 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0488 expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.655260:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.655262:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.655265:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc 0) 00000100:00100000:2.0:1788572547.655388:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360539392:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1398us (1803us total) trans 4294970785 rc 0/0 00000100:00100000:2.0:1788572547.655397:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 970 00000100:00100000:3.0:1788572547.655548:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35949:x1875453360539136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 955us (2007us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.655555:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4664 00000100:00100000:0.0:1788572547.661478:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.661533:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.662170:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360539648 00000100:00100000:3.0:1788572547.662190:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360539648 00000100:00100000:3.0:1788572547.662253:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4665 00000100:00100000:3.0:1788572547.662264:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360539648:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572547.662289:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.662313:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16451f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0496 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.662390:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.662405:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.662417:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.662447:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.662455:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.662461:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.662812:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360539776 00000100:00100000:1.0:1788572547.662817:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360539776 00000100:00100000:1.0:1788572547.662860:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4666 00000100:00100000:1.0:1788572547.662869:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35949:x1875453360539776:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.662881:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.662884:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572547.662904:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164518 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0488 expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.662934:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572547.662939:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572547.662948:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164534 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f048f expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572547.663376:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x59:0x0], index 1: 0 00010000:00010000:1.0:1788572547.663584:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16453b lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.663595:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16453b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.663604:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16453b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.663661:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb164534 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f048f expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.663677:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16453b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f048f expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.663685:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9627c8ed8300) 00010000:00010000:1.0:1788572547.663688:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572547.663691:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:1.0:1788572547.663768:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35949:x1875453360539776:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 900us (2233us total) trans 4294970786 rc 0/0 00000100:00100000:1.0:1788572547.663775:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4666 00010000:00010000:3.0:1788572547.664845:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb164549 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665127:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb164549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665137:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb164549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.665172:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xab:0x0]: returning size 3 00000004:00200000:3.0:1788572547.665176:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.665336:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb164549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665348:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb164549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665357:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665363:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665373:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665378:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.665383:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665390:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665402:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.665405:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.665409:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665415:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164526 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665427:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665442:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665449:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665454:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.665456:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665467:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665474:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xe44515a3 (ffff9626c1d8f100) 00010000:00010000:3.0:1788572547.665476:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.665479:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665484:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb16452d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665523:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb16451f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0496 expref: 180 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665535:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c400/0x85ace67feb164549 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0496 expref: 179 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.665542:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9626c1d8fb00) 00010000:00010000:3.0:1788572547.665544:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572547.665546:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14c400, rc 0) 00000100:00100000:3.0:1788572547.665649:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360539648:12345-192.168.202.40@tcp:101:diff.0 Request processed in 3386us (4181us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.665657:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4665 00000100:00100000:0.0:1788572547.667613:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.667702:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.668618:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360539904 00000100:00100000:2.0:1788572547.668631:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360539904 00000100:00100000:2.0:1788572547.668726:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360540160 00000100:00100000:2.0:1788572547.668730:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360540160 00000100:00100000:3.0:1788572547.668772:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4667 00000100:00100000:3.0:1788572547.668784:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35949:x1875453360539904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.668807:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.668829:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164550 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f049d expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669115:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669130:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669141:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669156:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669162:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669168:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669496:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669506:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669514:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.669543:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572547.669546:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.669566:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669583:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669590:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669597:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669604:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.669609:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669617:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669630:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572547.669633:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.669639:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669646:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164557 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669660:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669666:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669672:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669678:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14ce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.669681:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669687:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669693:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8f800) 00010000:00010000:3.0:1788572547.669695:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.669698:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669703:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16455e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669734:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164550 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 92 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669747:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 91 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.669754:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.669757:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.669759:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d200, rc 0) 00000100:00100000:3.0:1788572547.669819:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35949:x1875453360539904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1038us (2216us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.669826:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4667 00000100:00100000:3.0:1788572547.669868:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4668 00000100:00100000:3.0:1788572547.669878:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35947:x1875453360540160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.670637:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.670659:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1644c4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0473 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.670674:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572547.670682:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572547.670729:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.670740:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.670747:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 4/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.670781:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 89 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.670794:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 89 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.670808:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382388096:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572547.671053:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982de40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.672000:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.672587:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360540288 00000100:00100000:3.0:1788572547.672596:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360540288 00000100:00100000:3.0:1788572547.673237:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1310 00000100:00100000:3.0:1788572547.673251:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+42:2347:x1875453360540288:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572547.673270:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.673290:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4b54752/0x85ace67feb164573 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04a4 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572547.673316:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x58:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572547.673334:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=895s v=(0 0) 00010000:00010000:3.0:1788572547.673338:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572547.673348:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c4b54752/0x85ace67feb164573 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04a4 expref: 43 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.673365:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c4b54752/0x85ace67feb164573 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04a4 expref: 43 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.673377:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc600, rc 0) 00000100:00100000:3.0:1788572547.673429:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2347:x1875453360540288:12345-192.168.202.40@tcp:101:diff.0 Request processed in 183us (1433us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.673446:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1310 00000100:00100000:0.0:1788572547.674473:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.674635:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382388096:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.674660:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.674744:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360540416 00000100:00100000:2.0:1788572547.674751:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360540416 00010000:00010000:2.0:1788572547.674781:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 90 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.674855:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1106 00000100:00100000:2.0:1788572547.674865:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:2350:x1875453360540416:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572547.674878:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.674885:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 90 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.674898:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 90 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.674929:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb164565 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f049d expref: 90 pid: 5838 timeout: 995 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.674944:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:2.0:1788572547.674949:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.674958:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.674965:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.674998:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572547.675067:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:2350:x1875453360540416:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 203us (598us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.675074:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1106 00010000:00010000:0.0:1788572547.675129:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675137:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675159:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675169:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675178:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675426:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675451:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675458:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675464:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.675470:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675478:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675488:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.675490:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.675493:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675500:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb16456c lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675509:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675514:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675520:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675524:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.675526:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675531:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675538:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x593c98ad (ffff9627fb1e7800) 00010000:00010000:0.0:1788572547.675540:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.675542:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.675547:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16457a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x593c98ad bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.675601:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35947:x1875453360540160:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 5724us (7897us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572547.675609:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4668 00000100:00100000:0.0:1788572547.677679:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.677719:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360540672 00000100:00100000:0.0:1788572547.677728:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360540672 00000100:00100000:0.0:1788572547.677785:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4669 00000100:00100000:0.0:1788572547.677795:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360540672:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572547.677817:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.677839:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164581 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04ab expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.678327:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.678351:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360540800 00000100:00100000:1.0:1788572547.678356:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360540800 00000100:00100000:1.0:1788572547.678385:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4670 00000100:00100000:1.0:1788572547.678392:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:35947:x1875453360540800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.678401:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.678415:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb164588 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04b2 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.678454:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.678468:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.678480:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.678581:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb164588 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.678605:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 89 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.678613:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.678617:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572547.678620:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f200, rc 0) 00000100:00100000:1.0:1788572547.678672:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35947:x1875453360540800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 280us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.678680:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4670 00010000:00010000:0.0:1788572547.679693:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb164596 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.679710:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb164596 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.679720:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb164596 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.679732:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xab:0x0]: returning size 3 00010000:00010000:0.0:1788572547.679881:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb164581 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ab expref: 182 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.679899:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb164596 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ab expref: 181 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.679907:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9626c1d8fb00) 00010000:00010000:0.0:1788572547.679971:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572547.679976:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc200, rc 0) 00000100:00100000:0.0:1788572547.680051:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360540672:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2257us (2383us total) trans 4294970787 rc 0/0 00000100:00100000:0.0:1788572547.680059:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4669 00000100:00100000:1.0:1788572547.681442:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.681511:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.682505:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360541056 00000100:00100000:0.0:1788572547.682521:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360541056 00000100:00100000:0.0:1788572547.682579:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360541312 00000100:00100000:0.0:1788572547.682583:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360541312 00000100:00100000:0.0:1788572547.682615:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4671 00000100:00100000:0.0:1788572547.682625:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35947:x1875453360541056:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.682731:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.682749:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.682761:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.682990:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4672 00000100:00100000:1.0:1788572547.683018:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35948:x1875453360541312:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.683046:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.683068:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb1644f5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0481 expref: 90 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.683085:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9626c1d8ff00) 00010000:00010000:1.0:1788572547.683099:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572547.683115:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.683124:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.683131:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.683146:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.683153:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.683158:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572547.683691:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].74, 88 off14296 00000004:00080000:0.0:1788572547.683702:0:9040:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 895752696600ns, next at 896752693748ns, rc = 0 00000004:00010000:0.0:1788572547.683795:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000de9477d2 reply state 0000000091944f5a transno 4294970788 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.683805:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1788572547.684365:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684374:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684380:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684386:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.684391:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684399:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684410:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572547.684413:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.684417:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684423:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb1645ab lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684444:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684451:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684456:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684461:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.684463:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684469:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684476:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xdefaa6d5 (ffff9627c8ed8000) 00010000:00010000:1.0:1788572547.684478:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.684480:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.684485:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb1645b2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0xdefaa6d5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.684574:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35948:x1875453360541312:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1572us (3059us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572547.684582:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4672 00000100:00100000:1.0:1788572547.684651:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.685118:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35947:x1875453360541056:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2493us (3690us total) trans 4294970788 rc 0/0 00000100:00100000:0.0:1788572547.685125:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4671 00000100:00100000:3.0:1788572547.686159:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360541440 00000100:00100000:3.0:1788572547.686170:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360541440 00000100:00100000:3.0:1788572547.686241:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 317 00000100:00100000:3.0:1788572547.686252:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:2350:x1875453360541440:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.686267:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.686288:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb164596 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ab expref: 181 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.686304:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xab:0x0].0x0 (ffff9626c1d8fb00) 00010000:00010000:3.0:1788572547.686309:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.686343:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:2350:x1875453360541440:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 93us (1692us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.686350:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 317 00000100:00100000:1.0:1788572547.687515:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.687590:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360541568 00000100:00100000:3.0:1788572547.687598:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360541568 00002000:00100000:3.0:1788572547.687641:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x58:0x0] (0->4095) req@ffff9627c84ef100 x1875453360541568/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:208/0 lens 488/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572547.687651:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ef100 with x1875453360541568 ext(0->4095) 00002000:00010000:3.0:1788572547.687657:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84ef100 00000100:00100000:3.0:1788572547.687693:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 293 00000100:00100000:3.0:1788572547.687702:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2350:x1875453360541568:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572547.687948:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2350:x1875453360541568:12345-192.168.202.40@tcp:3:diff.0 Request processed in 246us (438us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572547.687955:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 293 00002000:00100000:3.0:1788572547.687961:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x58:0x0] (0->4095) req@ffff9627c84ef100 x1875453360541568/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:208/0 lens 488/4536 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572547.687968:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ef100 with x1875453360541568 ext(0->4095) 00002000:00010000:3.0:1788572547.687971:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84ef100 00000100:00100000:1.0:1788572547.689637:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572547.689669:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.689680:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.689688:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.689693:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.689698:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.689707:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.689723:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.689726:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572547.689730:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.689737:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1645a4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.690841:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.691307:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360541696 00000100:00100000:3.0:1788572547.691317:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360541696 00000100:00100000:3.0:1788572547.691418:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360541952 00000100:00100000:3.0:1788572547.691423:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360541952 00000100:00100000:1.0:1788572547.691501:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4673 00000100:00100000:1.0:1788572547.691514:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35948:x1875453360541696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.691538:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.691562:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1645b9 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04b9 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691605:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691619:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691630:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691744:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb1645b9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 92 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691763:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 91 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691771:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572547.691774:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572547.691777:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:1.0:1788572547.691863:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35948:x1875453360541696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 351us (2232us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.691870:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4673 00000100:00100000:1.0:1788572547.691890:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4674 00000100:00100000:1.0:1788572547.691898:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35949:x1875453360541952:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.691928:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.691944:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb16453b lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f048f expref: 90 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691961:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9627c8ed8300) 00010000:00010000:1.0:1788572547.691969:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572547.691986:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.691996:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692028:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692043:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692051:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692058:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692280:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692288:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692294:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692300:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.692305:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692312:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692321:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572547.692324:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.692327:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692333:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb1645c7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692342:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692347:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692353:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692357:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6a6800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.692359:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692365:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692370:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x93ebaf6d (ffff9627d1184000) 00010000:00010000:1.0:1788572547.692373:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.692375:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.692384:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6a6800/0x85ace67feb1645ce lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x93ebaf6d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.692448:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35949:x1875453360541952:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 549us (1610us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572547.692455:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4674 00000100:00100000:1.0:1788572547.694607:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.694676:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360542080 00000100:00100000:0.0:1788572547.694685:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360542080 00000100:00100000:0.0:1788572547.694859:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4675 00000100:00100000:0.0:1788572547.694870:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:35947:x1875453360542080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.694893:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.694911:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04c0 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.694969:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.694982:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.695039:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:0.0:1788572547.695043:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572547.695047:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bde00, rc 0) 00000100:00100000:0.0:1788572547.695099:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:35947:x1875453360542080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 231us (500us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.695105:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4675 00000100:00100000:2.0:1788572547.700788:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.701110:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.701327:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360542464 00000100:00100000:1.0:1788572547.701338:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360542464 00000100:00100000:1.0:1788572547.701575:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360542208 00000100:00100000:1.0:1788572547.701580:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360542208 00000100:00100000:3.0:1788572547.701620:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4676 00000100:00100000:3.0:1788572547.701632:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:35948:x1875453360542464:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.701737:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.701756:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.701769:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572547.702222:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].77, 88 off14488 00000004:00010000:3.0:1788572547.702311:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006257e898 reply state 00000000ea111637 transno 4294970789 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.702321:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572547.702446:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35948:x1875453360542464:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 805us (1659us total) trans 4294970789 rc 0/0 00000100:00100000:3.0:1788572547.702455:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4676 00000100:00100000:3.0:1788572547.702477:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4677 00000100:00100000:3.0:1788572547.702485:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+92:35949:x1875453360542208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.702497:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.702520:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645e3 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04c7 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.702546:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.702559:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.702567:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.702637:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645e3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 94 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.702654:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 93 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.702663:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572547.702666:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572547.702670:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572547.702704:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35949:x1875453360542208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 220us (1594us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.702711:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4677 00010000:00010000:1.0:1788572547.708347:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.708364:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.708372:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.708378:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.708394:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.708403:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.708421:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572547.708425:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572547.708430:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.708452:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1645dc lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.711895:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.712415:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360542848 00000100:00100000:0.0:1788572547.712426:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360542848 00000100:00100000:0.0:1788572547.712503:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1311 00000100:00100000:0.0:1788572547.712514:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360542848:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.713693:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360542848:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 1174us (1802us total) trans 4294967656 rc 0/0 00000100:00100000:0.0:1788572547.713704:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1311 00000100:00100000:2.0:1788572547.719448:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.719483:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.719569:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.719612:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360542720 00000100:00100000:0.0:1788572547.719621:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360542720 00000100:00100000:0.0:1788572547.719674:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 971 00000100:00100000:0.0:1788572547.719684:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:35867:x1875453360542720:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572547.720092:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360542720:12345-192.168.202.40@tcp:35:diff.0 Request processed in 408us (714us total) trans 4294970790 rc 0/0 00000100:00100000:0.0:1788572547.720100:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 971 00000100:00100000:3.0:1788572547.721036:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360543104 00000100:00100000:3.0:1788572547.721046:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360543104 00000100:00100000:3.0:1788572547.721090:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360543232 00000100:00100000:3.0:1788572547.721093:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360543232 00000100:00100000:3.0:1788572547.721113:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4678 00000100:00100000:3.0:1788572547.721122:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35949:x1875453360543104:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.721272:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4679 00000100:00100000:1.0:1788572547.721281:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+93:35948:x1875453360543232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.721294:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.721313:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04ce expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.721363:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 94 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.721379:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 94 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.721407:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572547.721411:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572547.721414:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96000, rc 0) 00000100:00100000:1.0:1788572547.721470:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35948:x1875453360543232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 190us (1901us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.721477:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4679 00010000:00010000:3.0:1788572547.722174:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.722284:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.722299:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572547.724170:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].75, 88 off14384 00000004:00010000:3.0:1788572547.724273:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aaa756a1 reply state 000000005e5dfea3 transno 4294970791 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.724285:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572547.724513:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35949:x1875453360543104:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 3387us (5024us total) trans 4294970791 rc 0/0 00000100:00100000:3.0:1788572547.724522:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4678 00000100:00100000:3.0:1788572547.729489:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.729580:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360543360 00000100:00100000:2.0:1788572547.729593:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360543360 00000100:00100000:2.0:1788572547.729652:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1312 00000100:00100000:2.0:1788572547.729661:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360543360:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.729971:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360543360:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 311us (492us total) trans 4294967658 rc 0/0 00000100:00100000:2.0:1788572547.729980:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1312 00010000:00010000:1.0:1788572547.731318:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.731336:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.731348:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.731353:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.731358:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.731367:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.731384:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:1.0:1788572547.731387:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572547.731391:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.731397:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb1645f8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.735365:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.735478:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.735540:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.735647:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360543744 00000100:00100000:0.0:1788572547.735657:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360543744 00000100:00100000:0.0:1788572547.735713:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360543872 00000100:00100000:0.0:1788572547.735717:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360543872 00000100:00100000:0.0:1788572547.735744:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4680 00000100:00100000:0.0:1788572547.735755:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+94:35949:x1875453360543744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.735780:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.735802:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04d5 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.735878:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.735895:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 95 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.735917:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:0.0:1788572547.735921:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572547.735925:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bce00, rc 0) 00000100:00100000:0.0:1788572547.735983:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+95:35949:x1875453360543744:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 230us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.735990:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4680 00000100:00100000:0.0:1788572547.736041:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4681 00000100:00100000:0.0:1788572547.736049:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360543872:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572547.736062:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.736072:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164606 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04dc expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736123:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736132:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736140:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736153:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736159:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736165:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736577:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736587:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736594:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572547.736624:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaa:0x0]: returning size 3 00000004:00200000:0.0:1788572547.736629:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572547.736647:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736657:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736666:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736671:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736678:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736683:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bdc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.736687:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736694:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736705:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572547.736707:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.736711:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736718:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16460d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736731:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736742:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736747:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736752:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.736755:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736761:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736768:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x2c9ecea3 (ffff9627fb1e7800) 00010000:00010000:0.0:1788572547.736770:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.736773:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736779:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164614 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736810:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb164606 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 183 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736824:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 182 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.736831:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:0.0:1788572547.736833:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572547.736836:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc 0) 00000100:00100000:0.0:1788572547.736868:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360543872:12345-192.168.202.40@tcp:101:diff.0 Request processed in 821us (1328us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.736876:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4681 00000100:00100000:0.0:1788572547.739352:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.739574:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360543616 00000100:00100000:3.0:1788572547.739584:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360543616 00000100:00100000:3.0:1788572547.739641:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360544128 00000100:00100000:3.0:1788572547.739645:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360544128 00000100:00100000:3.0:1788572547.739661:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 972 00000100:00100000:3.0:1788572547.739669:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35947:x1875453360543616:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.740243:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd0050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35947:x1875453360543616:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 572us (4887us total) trans 4294970792 rc 0/0 00000100:00100000:3.0:1788572547.740252:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 972 00000100:00100000:3.0:1788572547.740304:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 973 00000100:00100000:3.0:1788572547.740312:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+96:35948:x1875453360544128:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.740491:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35948:x1875453360544128:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 179us (1147us total) trans 4294970793 rc 0/0 00000100:00100000:3.0:1788572547.740498:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 973 00000100:00100000:0.0:1788572547.747103:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.747103:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.747189:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360544768 00000100:00100000:3.0:1788572547.747200:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360544768 00000100:00100000:3.0:1788572547.747268:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4682 00000100:00100000:3.0:1788572547.747279:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360544768:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572547.747302:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.747325:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164622 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04e3 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747454:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747473:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747486:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572547.747499:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaa:0x0]: returning size 3 00010000:00010000:3.0:1788572547.747609:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747618:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747625:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747631:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.747635:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747643:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747658:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:3.0:1788572547.747661:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572547.747666:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747673:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164629 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747727:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164622 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04e3 expref: 184 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747739:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572547.747741:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572547.747744:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164622 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04e3 expref: 184 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.747755:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572547.747847:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+183:35867:x1875453360544768:12345-192.168.202.40@tcp:101:diff.0 Request processed in 569us (752us total) trans 4294970794 rc 301/301 00000100:00100000:3.0:1788572547.747856:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4682 00000100:00100000:0.0:1788572547.748050:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.748238:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360544512 00000100:00100000:3.0:1788572547.748244:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360544512 00000100:00100000:3.0:1788572547.748283:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4683 00000100:00100000:3.0:1788572547.748292:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35947:x1875453360544512:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.748357:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35947:x1875453360544512:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 66us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.748364:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4683 00000100:00100000:2.0:1788572547.751191:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360544256 00000100:00100000:2.0:1788572547.751207:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360544256 00000100:00100000:3.0:1788572547.751540:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1313 00000100:00100000:3.0:1788572547.751552:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360544256:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.752249:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360544256:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 695us (5150us total) trans 4294967657 rc 0/0 00000100:00100000:3.0:1788572547.752259:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1313 00000100:00100000:1.0:1788572547.753795:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.753841:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.753869:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.753877:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360545024 00000100:00100000:1.0:1788572547.753885:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360545152 00000100:00100000:0.0:1788572547.753887:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360545024 00000100:00100000:1.0:1788572547.753887:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360545152 00000100:00100000:0.0:1788572547.753929:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360545280 00000100:00100000:0.0:1788572547.753932:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360545280 00000100:00100000:1.0:1788572547.753936:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4684 00000100:00100000:1.0:1788572547.753945:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35948:x1875453360545024:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.753949:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4685 00000100:00100000:0.0:1788572547.753955:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+183:35867:x1875453360545152:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572547.753968:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.753990:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04ea expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.754061:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.754065:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35948:x1875453360545024:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 121us (279us total) trans 0 rc 0/0 00010000:00010000:0.0:1788572547.754073:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 4/0,0 mode: --/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 184 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.754076:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4684 00010000:00010000:0.0:1788572547.754091:0:5839:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.754092:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4686 00000100:00100000:1.0:1788572547.754098:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+97:35947:x1875453360545280:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.754103:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 3/0,0 mode: --/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 184 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.754104:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572547.754113:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb16463e lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04f1 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.754121:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:0.0:1788572547.754124:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572547.754128:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 3/0,0 mode: --/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 184 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.754136:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd400, rc 0) 00010000:00010000:1.0:1788572547.754156:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164645 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.754157:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360545152:12345-192.168.202.40@tcp:101:diff.0 Request processed in 202us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.754163:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4685 00010000:00010000:1.0:1788572547.754167:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164645 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.754178:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164645 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572547.754221:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572547.754240:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164645 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.754265:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb16463e lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04f1 expref: 99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.754280:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164645 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04f1 expref: 98 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.754287:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.754289:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572547.754292:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97600, rc 0) 00000100:00100000:1.0:1788572547.754325:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35947:x1875453360545280:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 227us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.754332:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4686 00000100:00100000:0.0:1788572547.757297:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.757376:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360545536 00000100:00100000:3.0:1788572547.757386:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360545536 00000100:00100000:3.0:1788572547.757461:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 974 00000100:00100000:3.0:1788572547.757472:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+98:35949:x1875453360545536:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.757965:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35949:x1875453360545536:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 493us (674us total) trans 4294970795 rc 0/0 00000100:00100000:3.0:1788572547.757973:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 974 00000100:00100000:1.0:1788572547.758409:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.758533:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360545664 00000100:00100000:2.0:1788572547.758540:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360545664 00000100:00100000:2.0:1788572547.758587:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4687 00000100:00100000:2.0:1788572547.758596:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+99:35948:x1875453360545664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.758612:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.758632:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16464c lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04f8 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.758698:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164653 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.758713:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164653 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.758725:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164653 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572547.758752:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572547.758767:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164653 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.758792:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb16464c lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04f8 expref: 101 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.758807:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164653 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04f8 expref: 100 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.758816:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:2.0:1788572547.759504:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572547.759510:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572547.759576:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35948:x1875453360545664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 971us (1158us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.759584:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4687 00000100:00100000:1.0:1788572547.764522:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.764613:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360545792 00000100:00100000:2.0:1788572547.764624:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360545792 00000100:00100000:2.0:1788572547.764693:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1314 00000100:00100000:2.0:1788572547.764705:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360545792:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.764728:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.764753:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f04ff expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572547.764782:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x59:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7383, blocks: 0x1 00010000:00010000:2.0:1788572547.764807:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=895s v=(0 0) 00010000:00010000:2.0:1788572547.764812:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572547.764823:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.764842:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 13 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.764856:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572547.764932:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360545792:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 230us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.764939:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1314 00000100:00100000:0.0:1788572547.766350:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.766502:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360546048 00000100:00100000:1.0:1788572547.766510:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360546048 00000100:00100000:1.0:1788572547.766561:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4688 00000100:00100000:3.0:1788572547.766702:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.766758:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360546176 00000100:00100000:0.0:1788572547.766762:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360546176 00000100:00100000:0.0:1788572547.766798:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1315 00000100:00100000:0.0:1788572547.766806:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360546176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.766822:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.766840:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0506 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572547.766864:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x58:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7383, blocks: 0x1 00010000:00010000:0.0:1788572547.766881:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=895s v=(0 0) 00010000:00010000:0.0:1788572547.766885:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.766893:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.766914:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.766928:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc200, rc 0) 00000100:00100000:0.0:1788572547.766977:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360546176:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 172us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.766984:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1315 00000100:00100000:1.0:1788572547.767537:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35949:x1875453360546048:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.767776:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+100:35949:x1875453360546048:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 246us (1431us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.767785:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4688 00000100:00100000:3.0:1788572547.769506:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.771753:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.777205:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360546432 00000100:00100000:0.0:1788572547.777225:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360546432 00000100:00100000:0.0:1788572547.777304:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360546688 00000100:00100000:0.0:1788572547.777308:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360546688 00000100:00100000:0.0:1788572547.777344:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4689 00000100:00100000:0.0:1788572547.777355:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35947:x1875453360546432:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.777562:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4690 00000100:00100000:3.0:1788572547.777584:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35948:x1875453360546688:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.778132:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35948:x1875453360546688:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 550us (6388us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.778141:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4690 00000100:00100000:2.0:1788572547.778698:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.778742:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360546816 00000100:00100000:3.0:1788572547.778748:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360546816 00000100:00100000:3.0:1788572547.778786:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4691 00000100:00100000:3.0:1788572547.778793:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35949:x1875453360546816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.778809:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1788572547.778976:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35947:x1875453360546432:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 1621us (9590us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.778985:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4689 00010000:00010000:3.0:1788572547.779780:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164668 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f050d expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.779865:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16466f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.779880:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16466f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.779893:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16466f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.781495:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.781644:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360546944 00000100:00100000:0.0:1788572547.781653:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360546944 00000100:00100000:0.0:1788572547.781705:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1316 00000100:00100000:0.0:1788572547.781714:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35948:x1875453360546944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.781730:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572547.781745:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0514 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.781779:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.781788:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.781792:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.781797:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.781937:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.781951:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.781968:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382388224:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.782149:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2463000 going to sleep for 11 seconds 00000004:00200000:3.0:1788572547.782169:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572547.782207:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16466f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.782240:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164668 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f050d expref: 102 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.782258:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16466f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f050d expref: 101 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.782265:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572547.782268:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572547.782271:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc800, rc 0) 00000100:00100000:3.0:1788572547.782301:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35949:x1875453360546816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3508us (3605us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.782309:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4691 00000100:00100000:0.0:1788572547.789372:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.789540:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547072 00000100:00100000:2.0:1788572547.789550:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547072 00000100:00100000:2.0:1788572547.789690:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1317 00000100:00100000:2.0:1788572547.789701:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35947:x1875453360547072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.789725:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572547.789744:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f051b expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.789779:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 14 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.789787:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572547.789792:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572547.789798:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 4/0,0 mode: --/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.795459:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.795509:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.796118:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382388224:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.796159:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 14 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.796179:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bdc00, rc 0) 00000100:00100000:3.0:1788572547.796249:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35948:x1875453360546944:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 14535us (14760us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.796259:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1316 00000100:00100000:3.0:1788572547.796292:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547328 00000100:00100000:3.0:1788572547.796298:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547328 00000100:00100000:3.0:1788572547.796341:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1318 00000100:00100000:3.0:1788572547.796350:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360547328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.796365:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.796510:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0522 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572547.796536:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x59:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7383, blocks: 0x1 00010000:00010000:3.0:1788572547.796555:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=895s v=(0 0) 00010000:00010000:3.0:1788572547.796559:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572547.796569:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 16 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.796585:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 16 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.796595:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572547.796619:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360547328:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 270us (1109us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.796625:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1318 00010000:00010000:2.0:1788572547.797039:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 15 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.797053:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 15 pid: 10416 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.797068:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382388352:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572547.797127:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582af00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.797201:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547200 00000100:00100000:0.0:1788572547.797206:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547200 00010000:00010000:0.0:1788572547.797231:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.797267:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1107 00000100:00100000:0.0:1788572547.797274:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360547200:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.797288:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.797306:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.797319:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.797341:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb16465a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ff expref: 14 pid: 6681 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.797362:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.797367:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.797375:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.797383:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.797415:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 13 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.797425:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382388480:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572547.797467:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2463000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.807768:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.807819:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.807861:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547712 00000100:00100000:1.0:1788572547.807873:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547712 00010000:00010000:1.0:1788572547.807935:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 16 pid: 10416 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.807984:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1108 00000100:00100000:1.0:1788572547.807995:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2348:x1875453360547712:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.808049:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.808078:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 16 pid: 10416 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.808096:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 16 pid: 10416 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.808122:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.808127:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572547.808138:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.808148:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.808310:0:11655:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 16 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.808328:0:11655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382388608:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572547.808412:0:11655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627dffb3180 going to sleep for 11 seconds 00010000:00010000:2.0:1788572547.808730:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005cbd6cd/0x85ace67feb164661 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0506 expref: 16 pid: 10416 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.808747:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382388352:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572547.808778:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.808791:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00000100:00100000:2.0:1788572547.808825:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:35947:x1875453360547072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 19126us (19462us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.808833:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1317 00000100:00100000:3.0:1788572547.809251:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547584 00000100:00100000:3.0:1788572547.809259:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547584 00000100:00100000:3.0:1788572547.809307:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4692 00000100:00100000:3.0:1788572547.809317:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35949:x1875453360547584:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572547.809419:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35949:x1875453360547584:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 104us (1665us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.809427:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4692 00000100:00100000:0.0:1788572547.811992:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382388480:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572547.812059:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.812117:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360547200:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 14843us (16671us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.812128:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1107 00000100:00100000:1.0:1788572547.816263:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.816468:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547840 00000100:00100000:3.0:1788572547.816478:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547840 00000100:00100000:2.0:1788572547.816501:0:11655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:11655:1875453382388608:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572547.816523:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:3.0:1788572547.816527:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x59:0x0] (0->2) req@ffff9627dfae5880 x1875453360547840/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572547.816537:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae5880 with x1875453360547840 ext(0->2) 00002000:00010000:3.0:1788572547.816542:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627dfae5880 00000100:00100000:3.0:1788572547.816575:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 294 00000100:00100000:3.0:1788572547.816585:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360547840:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.816591:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360547712:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8599us (8771us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.816599:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1108 00000100:00100000:3.0:1788572547.817293:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360547840:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 706us (1038us total) trans 4294967659 rc 0/0 00000100:00100000:3.0:1788572547.817302:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 294 00002000:00100000:3.0:1788572547.817309:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x59:0x0] (0->2) req@ffff9627dfae5880 x1875453360547840/t4294967659(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/448 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572547.817318:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae5880 with x1875453360547840 ext(0->2) 00002000:00010000:3.0:1788572547.817322:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627dfae5880 00000100:00100000:1.0:1788572547.823930:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.824430:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360547968 00000100:00100000:3.0:1788572547.824452:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360547968 00000100:00100000:3.0:1788572547.824527:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1319 00000100:00100000:3.0:1788572547.824539:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:35949:x1875453360547968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.824563:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572547.824582:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0529 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.824617:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 15 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.824627:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572547.824632:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572547.824637:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 4/0,0 mode: --/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 15 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.824673:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 15 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.824688:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 15 pid: 6679 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.824703:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382388736:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572547.824850:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.829065:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.829118:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.829215:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548096 00000100:00100000:1.0:1788572547.829227:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548096 00002000:00100000:1.0:1788572547.829280:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x58:0x0] (0->2) req@ffff9627dfdbf800 x1875453360548096/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.829291:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbf800 with x1875453360548096 ext(0->2) 00002000:00010000:1.0:1788572547.829297:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbf800 00000100:00100000:1.0:1788572547.829333:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 295 00000100:00100000:1.0:1788572547.829343:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360548096:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.829392:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548224 00000100:00100000:2.0:1788572547.829397:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548224 00000100:00100000:2.0:1788572547.829451:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4693 00000100:00100000:2.0:1788572547.829458:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35948:x1875453360548224:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.829799:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360548096:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 456us (746us total) trans 4294967658 rc 0/0 00000100:00100000:1.0:1788572547.829808:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 295 00002000:00100000:1.0:1788572547.829816:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x58:0x0] (0->2) req@ffff9627dfdbf800 x1875453360548096/t4294967658(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/448 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.829824:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbf800 with x1875453360548096 ext(0->2) 00002000:00010000:1.0:1788572547.829828:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfdbf800 00000100:00100000:2.0:1788572547.839662:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.839743:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548352 00000100:00100000:0.0:1788572547.839751:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548352 00000100:00100000:1.0:1788572547.839762:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382388736:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.839788:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 18 pid: 6679 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.839791:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 18 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.839809:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:1.0:1788572547.839878:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:35949:x1875453360547968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 15340us (15956us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.839887:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1319 00000100:00100000:0.0:1788572547.840533:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1109 00000100:00100000:0.0:1788572547.840556:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca9d8450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+17:2350:x1875453360548352:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.840583:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.840612:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 17 pid: 6679 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.840626:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 17 pid: 6679 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.840652:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8402913/0x85ace67feb164684 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0522 expref: 17 pid: 6679 timeout: 995 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.840666:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.840670:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572547.840679:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.840695:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.840730:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 16 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.840745:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382388864:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572547.840809:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2463000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.844220:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.845633:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548480 00000100:00100000:0.0:1788572547.845646:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548480 00000100:00100000:0.0:1788572547.845712:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4694 00000100:00100000:0.0:1788572547.845723:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35947:x1875453360548480:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.845966:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35947:x1875453360548480:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 244us (1754us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.845973:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4694 00000100:00100000:3.0:1788572547.848526:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382388864:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572547.848557:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.848628:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca9d8450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360548352:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8085us (8976us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.848639:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1109 00000100:00100000:3.0:1788572547.852625:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.852856:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548608 00000100:00100000:1.0:1788572547.852867:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548608 00002000:00100000:1.0:1788572547.852930:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x59:0x0] (0->2) req@ffff9626c2c4bb80 x1875453360548608/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/0 e 0 to 0 dl 1788572558 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.852941:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2c4bb80 with x1875453360548608 ext(0->2) 00002000:00010000:1.0:1788572547.852948:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2c4bb80 00000100:00100000:1.0:1788572547.852983:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 296 00000100:00100000:1.0:1788572547.852993:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2350:x1875453360548608:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.853639:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360548608:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 644us (1018us total) trans 4294967659 rc 0/0 00000100:00100000:1.0:1788572547.853648:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 296 00002000:00100000:1.0:1788572547.853656:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x59:0x0] (0->2) req@ffff9626c2c4bb80 x1875453360548608/t4294967659(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:208/0 lens 496/448 e 0 to 0 dl 1788572558 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572547.853666:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2c4bb80 with x1875453360548608 ext(0->2) 00002000:00010000:1.0:1788572547.853669:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2c4bb80 00000100:00100000:1.0:1788572547.858855:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.858967:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548736 00000100:00100000:0.0:1788572547.858977:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548736 00000100:00100000:0.0:1788572547.859058:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1320 00000100:00100000:0.0:1788572547.859066:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360548736:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.863615:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.863653:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548864 00000100:00100000:0.0:1788572547.863664:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548864 00000100:00100000:0.0:1788572547.863723:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4695 00000100:00100000:0.0:1788572547.863734:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35949:x1875453360548864:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.863863:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+102:35949:x1875453360548864:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 129us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.863872:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4695 00000100:00100000:2.0:1788572547.867329:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.867425:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360548992 00000100:00100000:3.0:1788572547.867446:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360548992 00000100:00100000:3.0:1788572547.867506:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1321 00000100:00100000:3.0:1788572547.867517:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360548992:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:1.0:1788572547.890988:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+101:35948:x1875453360548224:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 61518us (61857us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.891028:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4693 00000001:00080000:2.0:1788572547.891267:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970790 is committed 00000001:00080000:2.0:1788572547.891279:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970788 is committed 00000004:00080000:2.0:1788572547.891283:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 40 committed 00000004:00080000:2.0:1788572547.891310:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 45 committed 00000020:00080000:2.0:1788572547.891348:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d0c12300 refcount 2 committed 0 result 0 batchid 141 00000020:00080000:2.0:1788572547.891353:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7ee00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d0c12f00 00000020:00080000:2.0:1788572547.891357:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7e200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4b65c00 00000020:00080000:2.0:1788572547.891370:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 138 batchid updated 00000001:00080000:2.0:1788572547.891377:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970795 is committed 00000001:00080000:2.0:1788572547.891382:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970794 is committed 00000020:00080000:2.0:1788572547.891411:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c982df00 refcount 2 committed 0 result 0 batchid 140 00000020:00080000:2.0:1788572547.891415:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceaf00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c982dd80 00000020:00080000:2.0:1788572547.891418:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceac00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ffd55400 00000020:00080000:2.0:1788572547.891424:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c982d540 refcount 2 committed 0 result 0 batchid 139 00000020:00080000:2.0:1788572547.891426:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2cea500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c982d0c0 00000020:00080000:2.0:1788572547.891429:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ceab00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4bd4400 00000020:00080000:2.0:1788572547.891470:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 141 committed batchid 138 00000020:00080000:2.0:1788572547.891660:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 138 00000100:00100000:1.0:1788572547.892400:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389248:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572547.892472:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.892505:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389760:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572547.892527:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.892550:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1788572547.892556:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389376:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1788572547.892576:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.892593:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389888:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572547.892612:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.892643:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389248 00000100:00100000:1.0:1788572547.892651:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389248 00000100:00100000:1.0:1788572547.892683:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389760 00000100:00100000:1.0:1788572547.892686:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389760 00000100:00100000:1.0:1788572547.892696:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389376 00000100:00100000:1.0:1788572547.892699:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389376 00000100:00100000:1.0:1788572547.892707:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389888 00000100:00100000:1.0:1788572547.892709:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389888 00000100:00100000:1.0:1788572547.892720:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1322 00000100:00100000:1.0:1788572547.892727:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3302:x1875453382389248:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:1.0:1788572547.892932:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:1.0:1788572547.892936:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb mtime from disk: 1788572517 -> 1788572517 00002000:00010000:1.0:1788572547.892939:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb atime from disk: 1788572517 -> 1788572517 00002000:00010000:1.0:1788572547.892941:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb ctime from disk: 1788572526 -> 1788572526 00002000:00010000:1.0:1788572547.892944:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x57:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:1.0:1788572547.892982:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382389248:12345-0@lo:2:osp-syn-1-0.0 Request processed in 254us (510us total) trans 4294967660 rc 0/0 00000100:00100000:1.0:1788572547.892989:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1322 00000100:00100000:1.0:1788572547.893037:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1323 00000100:00100000:1.0:1788572547.893044:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382389760:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1788572547.893147:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382389760:12345-0@lo:2:osp-syn-0-0.0 Request processed in 103us (618us total) trans 4294967660 rc 0/0 00000100:00100000:1.0:1788572547.893153:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1323 00000100:00100000:1.0:1788572547.893165:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1324 00000100:00100000:1.0:1788572547.893172:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3302:x1875453382389376:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572547.894346:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382388992:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572547.894398:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.894420:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382389504:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572547.894452:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.894469:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1788572547.894476:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382389120:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572547.894495:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.894505:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5b67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382389632:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572547.894521:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.894729:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382388992 00000100:00100000:2.0:1788572547.894737:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382388992 00000100:00100000:2.0:1788572547.894768:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389504 00000100:00100000:2.0:1788572547.894771:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389504 00000100:00100000:2.0:1788572547.894779:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389120 00000100:00100000:2.0:1788572547.894781:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389120 00000100:00100000:2.0:1788572547.894789:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382389632 00000100:00100000:2.0:1788572547.894792:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382389632 00000100:00100000:2.0:1788572547.894804:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1325 00000100:00100000:2.0:1788572547.894811:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382389888:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572547.894967:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572547.894970:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb mtime from disk: 1788572517 -> 1788572517 00002000:00010000:2.0:1788572547.894973:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb atime from disk: 1788572517 -> 1788572517 00002000:00010000:2.0:1788572547.894975:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb ctime from disk: 1788572526 -> 1788572526 00002000:00010000:2.0:1788572547.894978:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x58:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572547.895032:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+11:3302:x1875453382389888:12345-0@lo:2:osp-syn-0-0.0 Request processed in 220us (2418us total) trans 4294967661 rc 0/0 00000100:00100000:2.0:1788572547.895041:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1325 00000100:00100000:2.0:1788572547.895058:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1326 00000100:00100000:2.0:1788572547.895065:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382388992:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572547.895783:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382388992:12345-0@lo:2:osp-syn-1-0.0 Request processed in 716us (1381us total) trans 4294967661 rc 0/0 00000100:00100000:2.0:1788572547.895793:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1326 00000100:00100000:2.0:1788572547.895816:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1327 00000100:00100000:2.0:1788572547.895826:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382389504:12345-0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572547.895963:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382389504:12345-0@lo:2:osp-syn-0-0.0 Request processed in 140us (1510us total) trans 4294967662 rc 0/0 00000100:00100000:2.0:1788572547.895969:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1327 00000100:00100000:2.0:1788572547.895982:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1328 00000100:00100000:2.0:1788572547.895989:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382389120:12345-0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572547.896111:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382389120:12345-0@lo:2:osp-syn-1-0.0 Request processed in 123us (1614us total) trans 4294967662 rc 0/0 00000100:00100000:2.0:1788572547.896121:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1328 00000100:00100000:2.0:1788572547.896133:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1329 00000100:00100000:2.0:1788572547.896139:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+10:3304:x1875453382389632:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572547.896229:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572547.896232:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb mtime from disk: 1788572518 -> 1788572518 00002000:00010000:2.0:1788572547.896234:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb atime from disk: 1788572518 -> 1788572518 00002000:00010000:2.0:1788572547.896236:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb ctime from disk: 1788572526 -> 1788572526 00002000:00010000:2.0:1788572547.896238:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x56:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572547.896264:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+11:3304:x1875453382389632:12345-0@lo:2:osp-syn-0-0.0 Request processed in 125us (1742us total) trans 4294967663 rc 0/0 00000100:00100000:2.0:1788572547.896270:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1329 00002000:00010000:1.0:1788572547.896747:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572547.896751:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb mtime from disk: 0 -> 1788572547 00002000:00010000:1.0:1788572547.896753:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:1.0:1788572547.896756:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb ctime from disk: 1788572547 -> 1788572547 00002000:00010000:1.0:1788572547.896758:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572547.896795:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382389376:12345-0@lo:2:osp-syn-1-0.0 Request processed in 3623us (4217us total) trans 4294967663 rc 0/0 00000100:00100000:1.0:1788572547.896801:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1324 00000004:00080000:1.0:1788572547.896856:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b64a80/2, rc 0, transno 4294967663 00000100:00100000:1.0:1788572547.896870:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389376:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1788572547.896885:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b66300/2, rc 0, transno 4294967661 00000100:00100000:1.0:1788572547.896894:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389888:0@lo:2:osp-syn-0-0.0 00000004:00080000:1.0:1788572547.896907:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b64380/2, rc 0, transno 4294967660 00000100:00100000:1.0:1788572547.896910:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389248:0@lo:2:osp-syn-1-0.0 00000004:00080000:1.0:1788572547.896918:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b67b80/2, rc 0, transno 4294967660 00000100:00100000:1.0:1788572547.896921:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382389760:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572547.897596:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.897624:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360549120 00000100:00100000:2.0:1788572547.897630:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360549120 00000100:00100000:2.0:1788572547.897664:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1330 00000100:00100000:2.0:1788572547.897670:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360549120:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572547.901816:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f96d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3305:1875453382390144:0@lo:400:kworker.0 00000100:00100000:2.0:1788572547.901914:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572547.901966:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572547.901973:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3305:1875453382390016:0@lo:400:kworker.0 00000100:00100000:2.0:1788572547.902027:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.902481:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3302:1875453382390272:0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902513:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.902537:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788572547.902543:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382390400:0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902564:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572547.902594:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382390144 00000100:00100000:1.0:1788572547.902601:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382390144 00000100:00100000:1.0:1788572547.902643:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382390272 00000100:00100000:1.0:1788572547.902646:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382390272 00000100:00100000:1.0:1788572547.902673:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382390400 00000100:00100000:1.0:1788572547.902676:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382390400 00000100:00100000:1.0:1788572547.902699:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1788572547.902708:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1875453382390144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902771:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1875453382390144:12345-0@lo:400:kworker.0 Request processed in 63us (863us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.902779:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1788572547.902797:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:1.0:1788572547.902804:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1875453382390272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902835:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1875453382390272:12345-0@lo:400:kworker.0 Request processed in 32us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.902841:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:1.0:1788572547.902853:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:1.0:1788572547.902859:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382390400:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902886:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382390400:12345-0@lo:400:kworker.0 Request processed in 27us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.902892:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00080000:1.0:1788572547.902918:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572547.902957:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f96d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3305:1875453382390144:0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902986:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382390400:0@lo:400:kworker.0 00000100:00100000:1.0:1788572547.902995:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3302:1875453382390272:0@lo:400:kworker.0 00000004:00080000:3.0:1788572547.904031:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b64000/2, rc 0, transno 4294967661 00000100:00100000:3.0:1788572547.904056:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382388992:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1788572547.904084:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382390016 00000100:00100000:3.0:1788572547.904089:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382390016 00000100:00100000:3.0:1788572547.904138:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00000100:00100000:3.0:1788572547.904146:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0eff050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3305:x1875453382390016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572547.904570:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0eff050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3305:x1875453382390016:12345-0@lo:400:kworker.0 Request processed in 423us (2540us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.904578:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000004:00080000:3.0:1788572547.904626:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b67800/2, rc 0, transno 4294967662 00000100:00100000:3.0:1788572547.904645:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382389504:0@lo:2:osp-syn-0-0.0 00000004:00080000:3.0:1788572547.904661:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b64e00/2, rc 0, transno 4294967662 00000100:00100000:3.0:1788572547.904675:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b64e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382389120:0@lo:2:osp-syn-1-0.0 00000004:00080000:3.0:1788572547.904687:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f5b67480/2, rc 0, transno 4294967663 00000100:00100000:3.0:1788572547.904690:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5b67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382389632:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1788572547.904880:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3305:1875453382390016:0@lo:400:kworker.0 00000100:00100000:0.0:1788572547.913656:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+19:2350:x1875453360548736:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 54577us (54800us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.913673:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1320 00000100:00100000:0.0:1788572547.913743:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+18:2350:x1875453360548992:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 46227us (46420us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.913751:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1321 00000001:00080000:0.0:1788572547.913778:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967656 is committed 00000001:00080000:0.0:1788572547.913792:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967659 is committed 00000001:00080000:0.0:1788572547.913806:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967655 is committed 00000100:00100000:1.0:1788572547.916359:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.917610:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360549376 00000100:00100000:2.0:1788572547.917623:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360549376 00000100:00100000:2.0:1788572547.917688:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4696 00000100:00100000:2.0:1788572547.917697:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35949:x1875453360549376:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.917753:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572547.917777:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16466f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f050d expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.917795:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:2.0:1788572547.917800:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572547.917827:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.917841:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.917863:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572547.918065:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000024cb6b9a reply state 00000000bf5ce409 transno 4294970796 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.918075:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572547.918169:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35949:x1875453360549376:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 472us (1818us total) trans 4294970796 rc 0/0 00000100:00100000:2.0:1788572547.918176:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4696 00000100:00100000:1.0:1788572547.919882:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:1.0:1788572547.919950:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.919961:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.919967:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.919973:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.919978:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.919985:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.920000:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:1.0:1788572547.920028:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572547.920033:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.920039:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05c00/0x85ace67feb164692 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.920076:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360549632 00000100:00100000:1.0:1788572547.920087:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360549632 00000100:00100000:1.0:1788572547.920132:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4697 00000100:00100000:1.0:1788572547.920140:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35947:x1875453360549632:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.920201:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.920218:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164645 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04f1 expref: 65 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.920228:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.920232:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572547.920258:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.920272:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.920282:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572547.920447:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008acc9baf reply state 00000000ce0c2b1e transno 4294970797 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.920455:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572547.920549:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35947:x1875453360549632:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 408us (676us total) trans 4294970797 rc 0/0 00000100:00100000:1.0:1788572547.920556:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4697 00000100:00100000:2.0:1788572547.924386:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572547.924520:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.924534:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.924542:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.924548:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572547.924554:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.924563:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.924584:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:0.0:1788572547.924588:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572547.924592:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.924599:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb164699 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.924684:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360549760 00000100:00100000:1.0:1788572547.924696:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360549760 00000100:00100000:1.0:1788572547.924753:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1331 00000100:00100000:1.0:1788572547.924764:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360549760:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572547.928197:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:1.0:1788572547.928203:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:1.0:1788572547.928207:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:1.0:1788572547.928209:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb ctime from disk: 1788572547 -> 1788572526 00002000:00010000:1.0:1788572547.928211:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1788572547.934953:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360549760:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 10178us (10563us total) trans 4294967664 rc 0/0 00000100:00100000:1.0:1788572547.934968:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1331 00000100:00100000:2.0:1788572547.937788:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.939357:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360549888 00000100:00100000:0.0:1788572547.939373:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360549888 00000100:00100000:0.0:1788572547.939439:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1332 00000100:00100000:0.0:1788572547.939451:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360549888:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572547.939649:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:0.0:1788572547.939653:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb mtime from disk: 1788572547 -> 1788572517 00002000:00010000:0.0:1788572547.939655:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:0.0:1788572547.939657:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb ctime from disk: 1788572547 -> 1788572526 00002000:00010000:0.0:1788572547.939660:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x58:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:0.0:1788572547.939723:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360549888:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 274us (1946us total) trans 4294967665 rc 0/0 00000100:00100000:0.0:1788572547.939730:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1332 00000100:00100000:3.0:1788572547.948645:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.948692:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.948760:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360550144 00000100:00100000:2.0:1788572547.948770:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360550144 00000100:00100000:2.0:1788572547.948826:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360550400 00000100:00100000:2.0:1788572547.948829:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360550400 00000100:00100000:2.0:1788572547.948849:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4698 00000100:00100000:2.0:1788572547.948858:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35949:x1875453360550144:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572547.948949:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.948966:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.948975:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.949041:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.949055:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 66 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.949070:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382390528:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572547.949145:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b86c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.949933:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4699 00000100:00100000:3.0:1788572547.949947:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:35947:x1875453360550400:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.949989:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.950000:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 66 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.950040:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.950068:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 66 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.950079:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 66 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.950090:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ec380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382390656:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572547.950138:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.951686:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360549120:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 54008us (54086us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.951698:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1330 00000001:00080000:3.0:1788572547.951747:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967660 is committed 00000001:00080000:3.0:1788572547.951758:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967654 is committed 00000001:00080000:3.0:1788572547.951764:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967663 is committed 00000001:00080000:3.0:1788572547.951773:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967658 is committed 00000001:00080000:3.0:1788572547.951780:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967659 is committed 00000100:00100000:1.0:1788572547.952868:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ec380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382390656:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572547.952896:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.953181:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382390528:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.953258:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.957325:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.957577:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.957738:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.957870:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360550912 00000100:00100000:1.0:1788572547.957881:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360550912 00000100:00100000:1.0:1788572547.957951:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4700 00000100:00100000:1.0:1788572547.957962:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:35948:x1875453360550912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572547.958050:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.958078:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164653 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04f8 expref: 67 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.958098:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572547.958103:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572547.958134:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.958149:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.958161:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572547.958343:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000040390cc7 reply state 00000000775a7da9 transno 4294970798 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.958353:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572547.958459:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:35948:x1875453360550912:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 498us (720us total) trans 4294970798 rc 0/0 00000100:00100000:1.0:1788572547.958466:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4700 00000100:00100000:3.0:1788572547.959833:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360550528 00000100:00100000:3.0:1788572547.959844:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360550528 00010000:00010000:3.0:1788572547.959883:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 68 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.959922:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360550656 00000100:00100000:3.0:1788572547.959926:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360550656 00010000:00010000:3.0:1788572547.959941:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 69 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.959971:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1110 00000100:00100000:3.0:1788572547.959980:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2349:x1875453360550528:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572547.959993:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572547.960024:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 69 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.960036:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 69 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.960055:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1645ea lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c7 expref: 69 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.960070:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572547.960075:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.960085:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.960093:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.960118:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572547.960163:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2349:x1875453360550528:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 185us (2849us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572547.960171:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1110 00000100:00100000:1.0:1788572547.960367:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1111 00000100:00100000:1.0:1788572547.960377:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360550656:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.960386:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.960391:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 67 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960401:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 67 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960415:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb16458f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa2:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b2 expref: 67 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960426:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.960430:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960449:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960455:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960472:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.960493:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360550656:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 119us (2916us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.960500:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1111 00010000:00010000:1.0:1788572547.960618:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960626:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960657:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960664:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960671:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960677:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74ccc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572547.960681:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960690:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960699:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572547.960701:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572547.960705:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.960710:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb1646a7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.960754:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:35947:x1875453360550400:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10812us (12061us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572547.960762:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4699 00010000:00010000:3.0:1788572547.961178:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961270:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961287:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961293:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961298:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961303:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.961306:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961312:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961322:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:3.0:1788572547.961325:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572547.961328:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.961335:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1646a0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.961384:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35949:x1875453360550144:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 12526us (12749us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572547.961393:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4698 00010000:00010000:3.0:1788572547.963747:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.963762:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.963768:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.963773:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.963778:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.963786:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.963800:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:3.0:1788572547.963803:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572547.963807:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.963814:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646ae lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.976249:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.976357:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551040 00000100:00100000:0.0:1788572547.976365:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551040 00000100:00100000:0.0:1788572547.976418:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1333 00000100:00100000:0.0:1788572547.976425:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360551040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572547.976691:0:10416:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:0.0:1788572547.976695:0:10416:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb mtime from disk: 0 -> 1788572518 00002000:00010000:0.0:1788572547.976697:0:10416:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb atime from disk: 0 -> 1788572518 00002000:00010000:0.0:1788572547.976699:0:10416:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb ctime from disk: 1788572547 -> 1788572526 00002000:00010000:0.0:1788572547.976700:0:10416:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1788572547.976762:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360551040:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 337us (521us total) trans 4294967664 rc 0/0 00000100:00100000:0.0:1788572547.976767:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1333 00000100:00100000:2.0:1788572547.981685:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.981730:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.982059:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572547.982151:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551552 00000100:00100000:3.0:1788572547.982162:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551552 00000100:00100000:3.0:1788572547.982226:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4701 00000100:00100000:3.0:1788572547.982236:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35948:x1875453360551552:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.982310:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.982328:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 63 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.982338:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.982375:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 63 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.982401:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 63 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.982416:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382390784:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572547.982506:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572547.982842:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551296 00000100:00100000:0.0:1788572547.982848:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551296 00000100:00100000:0.0:1788572547.982924:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551168 00000100:00100000:0.0:1788572547.982928:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551168 00000100:00100000:0.0:1788572547.982959:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1334 00000100:00100000:0.0:1788572547.982968:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360551296:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.983000:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/1,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.983034:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.983042:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x59:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1788572547.983046:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.983050:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 4/1,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.983073:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 14 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.983083:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 14 pid: 20911 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.983094:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382390912:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.983132:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572547.983201:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1335 00000100:00100000:3.0:1788572547.983210:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360551168:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572547.983232:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.983244:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.983252:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x58:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788572547.983255:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572547.983258:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.983278:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.983287:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 14 pid: 6681 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.983296:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382391040:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572547.983321:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d600 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.989162:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.989189:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.989218:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551808 00000100:00100000:1.0:1788572547.989227:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551808 00010000:00010000:1.0:1788572547.989267:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 15 pid: 20911 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.989302:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551680 00000100:00100000:1.0:1788572547.989306:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551680 00010000:00010000:1.0:1788572547.989321:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 64 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.989344:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1112 00000100:00100000:1.0:1788572547.989354:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360551808:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.989367:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.989565:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 15 pid: 20911 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989579:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 15 pid: 20911 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989605:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.989610:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572547.989616:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989627:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989648:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.989699:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360551808:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 346us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.989707:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1112 00000100:00100000:1.0:1788572547.989727:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1113 00000100:00100000:1.0:1788572547.989734:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360551680:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572547.989744:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572547.989751:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 64 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989763:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 64 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989778:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572547.989784:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989793:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989800:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.989812:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572547.989831:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360551680:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 97us (640us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572547.989838:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1113 00000100:00100000:1.0:1788572547.989884:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572547.990053:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360551936 00000100:00100000:0.0:1788572547.990058:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360551936 00010000:00010000:0.0:1788572547.990080:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 15 pid: 6681 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.990105:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1114 00000100:00100000:0.0:1788572547.990113:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360551936:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572547.990121:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572547.990133:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 15 pid: 6681 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990144:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 15 pid: 6681 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990156:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.990160:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572547.990163:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990171:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990190:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572547.990214:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360551936:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 103us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.990221:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1114 00010000:00010000:0.0:1788572547.990264:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000002eb965a/0x85ace67feb16468b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0529 expref: 14 pid: 20911 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.990278:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382390912:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572547.990296:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990306:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990314:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990337:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990346:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990353:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990362:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572547.990366:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990375:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990549:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.990559:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000059d071c1/0x85ace67feb1646bc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.990606:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360551296:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 7640us (8931us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572547.990614:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1334 00010000:00010000:3.0:1788572547.990711:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb1645c0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04b9 expref: 63 pid: 5837 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.990726:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382390784:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572547.990739:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990750:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990756:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990782:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990789:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990795:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990799:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572547.990803:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990809:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990818:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:3.0:1788572547.990820:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572547.990824:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572547.990829:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1646b5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572547.990867:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35948:x1875453360551552:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8632us (8809us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572547.990876:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4701 00010000:00010000:2.0:1788572547.991079:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2c053aa/0x85ace67feb16467d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x58:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f051b expref: 12 pid: 6681 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.991092:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ec380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382391040:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572547.991106:0:13272:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991115:0:13272:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991121:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991138:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991144:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991150:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991156:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572547.991158:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991165:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991175:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572547.991182:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c70070ee/0x85ace67feb1646c3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572547.991219:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360551168:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 8011us (9489us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572547.991227:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1335 00000100:00100000:1.0:1788572547.998775:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572547.998821:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.998830:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360552064 00000100:00100000:2.0:1788572547.998840:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360552064 00000100:00100000:1.0:1788572547.998869:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572547.998890:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1336 00000100:00100000:2.0:1788572547.998900:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360552064:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00000100:00100000:0.0:1788572547.998979:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360552320 00000100:00100000:0.0:1788572547.998985:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360552320 00000100:00100000:0.0:1788572547.999043:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360552576 00000100:00100000:0.0:1788572547.999047:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360552576 00000100:00100000:0.0:1788572547.999071:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4702 00000100:00100000:0.0:1788572547.999078:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:35947:x1875453360552320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.999154:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 3/0,1 mode: --/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.999175:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.999187:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572547.999420:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4703 00000100:00100000:1.0:1788572547.999428:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35949:x1875453360552576:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572547.999457:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.999468:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 64 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.999472:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.999478:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.999480:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572547.999488:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.999513:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 64 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572547.999529:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 64 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572547.999543:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382391168:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572547.999621:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627f582a3c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788572548.001067:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.001089:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.001096:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.001129:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.001139:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.001151:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382391296:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572548.001210:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572548.004830:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.004937:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382391296:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572548.004965:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.005114:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360552704 00000100:00100000:1.0:1788572548.005122:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360552704 00010000:00010000:1.0:1788572548.005159:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 66 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.005206:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1115 00000100:00100000:1.0:1788572548.005217:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360552704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572548.005231:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572548.005238:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 66 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.005250:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 66 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.005271:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572548.005276:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.005285:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.005292:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.005309:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572548.005354:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360552704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 139us (534us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.005361:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1115 00010000:00010000:3.0:1788572548.005670:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1645d5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04c0 expref: 65 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.005686:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382391168:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572548.005703:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.005710:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.005716:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.005819:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.005887:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360552832 00000100:00100000:0.0:1788572548.005893:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360552832 00010000:00010000:0.0:1788572548.005910:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.005939:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1116 00000100:00100000:0.0:1788572548.005947:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2350:x1875453360552832:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572548.005955:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572548.005960:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.005971:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.005984:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1645ff lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04d5 expref: 65 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.005995:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:0.0:1788572548.006000:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.006023:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.006029:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.006049:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572548.006080:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360552832:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 134us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.006087:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1116 00010000:00010000:0.0:1788572548.006152:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.006159:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572548.006231:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970799 00000001:00080000:0.0:1788572548.006250:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970800 00000001:00080000:3.0:1788572548.006382:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970801 00000001:00080000:3.0:1788572548.006449:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970803 00000100:00100000:3.0:1788572548.006532:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382391424:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572548.006656:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572548.006683:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572548.006702:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382391424 00000100:00100000:3.0:1788572548.006706:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382391424 00000100:00100000:3.0:1788572548.006744:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1788572548.006751:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:5837:x1875453382391424:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572548.006815:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:5837:x1875453382391424:12345-0@lo:1000:mdt00_000.0 Request processed in 64us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.006822:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1788572548.006920:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967655 gen 2 00000004:00080000:3.0:1788572548.006926:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967655 committed transno 4294967655 00000100:00100000:3.0:1788572548.006942:0:5837:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627f5b64380 x1875453382389248/t4294967660(4294967660) o2->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1788572563 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572548.006952:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382391424:0@lo:1000:mdt00_000.0 00000001:00080000:3.0:1788572548.007473:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970804 00000100:00100000:3.0:1788572548.007525:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382391552:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572548.007556:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572548.008262:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572548.008295:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382391552 00000100:00100000:1.0:1788572548.008302:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382391552 00000100:00100000:1.0:1788572548.008351:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:1.0:1788572548.008361:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:5837:x1875453382391552:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572548.008565:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:5837:x1875453382391552:12345-0@lo:1000:mdt00_000.0 Request processed in 204us (1006us total) trans 4294967666 rc 0/0 00000100:00100000:1.0:1788572548.008573:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00080000:1.0:1788572548.008593:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572548.008633:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382391552:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1788572548.008665:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008674:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008681:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008687:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.008691:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008698:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008711:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572548.008715:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572548.008719:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008724:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb1646d1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008762:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008768:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008773:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008778:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.008781:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008786:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008793:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:1.0:1788572548.008795:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572548.008802:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 177 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008813:0:5837:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 177 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008821:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 177 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.008863:0:5837:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd400/0x85ace67feb164637 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ea expref: 177 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.008880:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382391680:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572548.009181:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.009200:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.009207:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x59:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1788572548.009211:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572548.009216:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.009242:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.009256:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 11 pid: 6679 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.009270:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382391808:192.168.202.40@tcp:104: 00000001:00080000:0.0:1788572548.009762:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970802 00000001:00080000:0.0:1788572548.009848:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970806 00000100:00100000:0.0:1788572548.009933:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382391936:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1788572548.009973:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572548.010085:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd3900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572548.010281:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572548.010350:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8540 going to sleep for 11 seconds 00000100:00100000:3.0:1788572548.010514:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382391936 00000100:00100000:3.0:1788572548.010521:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382391936 00000100:00100000:3.0:1788572548.010571:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1788572548.010580:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:9040:x1875453382391936:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572548.010663:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:9040:x1875453382391936:12345-0@lo:1000:mdt00_003.0 Request processed in 82us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.010670:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:1.0:1788572548.010783:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382391936:0@lo:1000:mdt00_003.0 00000001:00080000:1.0:1788572548.011388:0:9040:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970807 00000100:00100000:1.0:1788572548.011467:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382392064:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1788572548.011505:0:9040:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572548.011527:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572548.011580:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382392064 00000100:00100000:3.0:1788572548.011585:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382392064 00000100:00100000:3.0:1788572548.011619:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1788572548.011627:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:9040:x1875453382392064:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1788572548.011805:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:9040:x1875453382392064:12345-0@lo:1000:mdt00_003.0 Request processed in 178us (298us total) trans 4294967667 rc 0/0 00000100:00100000:3.0:1788572548.011812:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:1.0:1788572548.011970:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:9040:1875453382392064:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1788572548.012074:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012084:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012091:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012096:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.012101:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012108:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012122:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9626c1d8f500) 00010000:00010000:1.0:1788572548.012125:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572548.012130:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012136:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1646df lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012164:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012169:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012174:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012178:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.012181:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012265:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012278:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9627ffc16a00) 00010000:00010000:1.0:1788572548.012281:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572548.012285:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012292:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb1600e6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012316:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012328:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012337:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012344:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012349:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012354:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012359:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.012362:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012368:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012376:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9627ffc16a00) 00010000:00010000:1.0:1788572548.012378:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.012381:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012387:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1646ed lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012396:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012402:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012409:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012414:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.012417:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012423:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012430:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9627ffc16a00) 00010000:00010000:1.0:1788572548.012443:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572548.012446:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.012452:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96200/0x85ace67feb1646d8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.012599:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35949:x1875453360552576:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 13170us (13727us total) trans 4294970808 rc 0/0 00000100:00100000:1.0:1788572548.012609:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4703 00000020:00080000:1.0:1788572548.012614:0:9040:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572548.016308:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.016418:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360552960 00000100:00100000:3.0:1788572548.016426:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360552960 00000100:00100000:3.0:1788572548.016489:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 975 00000100:00100000:3.0:1788572548.016498:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360552960:12345-192.168.202.40@tcp:49:diff.0 00000100:00100000:3.0:1788572548.016618:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360552960:12345-192.168.202.40@tcp:49:diff.0 Request processed in 121us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.016626:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 975 00000100:00100000:1.0:1788572548.017101:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382391680:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572548.017127:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.017138:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb15ff49 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.017167:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.017181:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 177 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.017200:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.017230:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 177 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.017242:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 177 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.017256:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382392192:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572548.017305:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0686c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572548.019473:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.019545:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360553216 00000100:00100000:1.0:1788572548.019553:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360553216 00000100:00100000:1.0:1788572548.019605:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 976 00000100:00100000:1.0:1788572548.019614:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:35949:x1875453360553216:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572548.020442:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa4:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572548.020552:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970809 00010000:00010000:1.0:1788572548.020598:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.020615:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.020628:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.020637:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.020644:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.020650:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.020718:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572548.020822:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:35949:x1875453360553216:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1206us (1356us total) trans 4294970810 rc 0/0 00000100:00100000:1.0:1788572548.020831:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 976 00000020:00080000:1.0:1788572548.020835:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788572548.022301:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572548.022314:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.022323:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572548.022328:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.022340:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.022357:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa4:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:2.0:1788572548.022360:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572548.022365:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.022371:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb1646fb lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa4:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.022394:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572548.022396:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572548.022397:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572548.024520:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.024602:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360553344 00000100:00100000:1.0:1788572548.024612:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360553344 00000100:00100000:1.0:1788572548.024664:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1337 00000100:00100000:1.0:1788572548.024675:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:35867:x1875453360553344:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.024693:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.024714:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0f574ee/0x85ace67feb164702 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0530 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572548.024743:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x58:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:1.0:1788572548.024767:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=896s v=(0 0) 00010000:00010000:1.0:1788572548.024772:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x58:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572548.024781:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c0f574ee/0x85ace67feb164702 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0530 expref: 54 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.024801:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0f574ee/0x85ace67feb164702 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x58:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0530 expref: 54 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.024813:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96200, rc 0) 00000100:00100000:1.0:1788572548.024873:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+54:35867:x1875453360553344:12345-192.168.202.40@tcp:101:diff.0 Request processed in 200us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.024880:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1337 00000100:00100000:0.0:1788572548.029602:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.029736:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382391808:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572548.029766:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.029860:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382392192:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572548.029878:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.029951:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.031601:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360553472 00000100:00100000:3.0:1788572548.031612:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360553472 00010000:00010000:3.0:1788572548.031677:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 12 pid: 6679 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.031720:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360553600 00000100:00100000:3.0:1788572548.031724:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360553600 00010000:00010000:3.0:1788572548.031743:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 178 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.031768:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1117 00000100:00100000:3.0:1788572548.031777:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360553472:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.031793:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.031816:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 12 pid: 6679 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.031829:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 12 pid: 6679 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.031864:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fde277ca/0x85ace67feb164676 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x59:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0514 expref: 12 pid: 6679 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.031882:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572548.031887:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572548.031893:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.032024:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1118 00000100:00100000:2.0:1788572548.032034:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:2349:x1875453360553600:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.032044:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.032051:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 178 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.032062:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 178 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.032076:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb16461b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaa:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04dc expref: 178 pid: 22536 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.032088:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:2.0:1788572548.032092:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.032101:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.032107:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.032132:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572548.032195:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:2349:x1875453360553600:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 151us (2233us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.032202:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1118 00010000:00010000:3.0:1788572548.033230:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033282:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.033324:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360553472:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1546us (3730us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.033333:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1117 00010000:00010000:3.0:1788572548.033368:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033377:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033406:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033415:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033423:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033429:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572548.033444:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033455:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033469:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.033477:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2c053aa/0x85ace67feb1646e6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.033527:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360552064:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 34629us (34760us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.033535:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1336 00000100:00080000:0.0:1788572548.034451:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1788572548.036118:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036134:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036149:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036156:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036162:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036168:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.036172:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036179:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036209:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:2.0:1788572548.036212:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.036216:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036223:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb1646f4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xaa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036238:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036248:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036253:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036257:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.036260:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036266:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036272:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaa:0x0].0x0 (ffff9627d55e6b00) 00010000:00010000:2.0:1788572548.036274:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572548.036277:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 2/0,0 mode: --/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.036281:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1646ca lrc: 0/0,0 mode: --/PW res: [0x200000401:0xaa:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.036502:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35947:x1875453360552320:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 37420us (37675us total) trans 4294970805 rc 0/0 00000100:00100000:2.0:1788572548.036514:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4702 00000020:00080000:2.0:1788572548.036519:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:0.0:1788572548.038666:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.038711:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.038779:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360553984 00000100:00100000:2.0:1788572548.038789:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360553984 00000100:00100000:2.0:1788572548.038845:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4704 00000100:00100000:2.0:1788572548.038865:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:35948:x1875453360553984:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572548.038958:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.038975:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.038986:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.039543:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.039556:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 73 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.039565:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.039602:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 73 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.039617:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 73 pid: 9040 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.039633:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382392320:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572548.039726:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8540 going to sleep for 11 seconds 00000100:00100000:3.0:1788572548.039819:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360553728 00000100:00100000:3.0:1788572548.039826:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360553728 00002000:00100000:3.0:1788572548.039864:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x58:0x0] (0->4095) req@ffff9627e7bc5180 x1875453360553728/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.039875:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7bc5180 with x1875453360553728 ext(0->4095) 00002000:00010000:3.0:1788572548.039880:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7bc5180 00000100:00100000:3.0:1788572548.039924:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 297 00000100:00100000:3.0:1788572548.039932:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+54:2350:x1875453360553728:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572548.040111:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+54:2350:x1875453360553728:12345-192.168.202.40@tcp:3:diff.0 Request processed in 179us (1454us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572548.040118:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 297 00002000:00100000:3.0:1788572548.040125:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x58:0x0] (0->4095) req@ffff9627e7bc5180 x1875453360553728/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.040132:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7bc5180 with x1875453360553728 ext(0->4095) 00002000:00010000:3.0:1788572548.040136:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7bc5180 00000100:00100000:0.0:1788572548.045748:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.045831:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360554240 00000100:00100000:2.0:1788572548.045841:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360554240 00000100:00100000:2.0:1788572548.045921:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 977 00000100:00100000:2.0:1788572548.045932:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+177:35867:x1875453360554240:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572548.046351:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360554240:12345-192.168.202.40@tcp:35:diff.0 Request processed in 419us (611us total) trans 4294970812 rc 0/0 00000100:00100000:2.0:1788572548.046359:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 977 00000100:00100000:0.0:1788572548.049152:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.049293:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360554496 00000100:00100000:1.0:1788572548.049305:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360554496 00000100:00100000:1.0:1788572548.049367:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 978 00000100:00100000:1.0:1788572548.049378:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fd3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:35947:x1875453360554496:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572548.049782:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa2:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572548.049824:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970813 00010000:00010000:1.0:1788572548.049871:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.049887:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.049898:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.049909:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.049915:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.049920:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.049936:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572548.050045:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fd3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:35947:x1875453360554496:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 668us (905us total) trans 4294970814 rc 0/0 00000100:00100000:1.0:1788572548.050053:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 978 00000020:00080000:1.0:1788572548.050058:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788572548.050721:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572548.050729:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.050737:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572548.050741:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.050749:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.050764:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa2:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:0.0:1788572548.050767:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572548.050772:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.050778:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb164717 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.050796:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572548.050798:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572548.050800:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572548.051378:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcf480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382392320:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572548.051401:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.056542:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.056697:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.056772:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360554752 00000100:00100000:1.0:1788572548.056782:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360554752 00010000:00010000:1.0:1788572548.056824:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 76 pid: 9040 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.057088:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1119 00000100:00100000:1.0:1788572548.057099:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2349:x1875453360554752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572548.057115:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572548.057122:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 76 pid: 9040 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.057135:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 76 pid: 9040 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.057162:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1645f1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f04ce expref: 76 pid: 9040 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.057177:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572548.057193:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.057202:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.057210:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.057239:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572548.057289:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360554752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 191us (591us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.057297:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1119 00010000:00010000:0.0:1788572548.058105:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.058116:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572548.058321:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970811 00000001:00080000:0.0:1788572548.058498:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970815 00000001:00080000:0.0:1788572548.058547:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970816 00000100:00100000:0.0:1788572548.058630:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382392448:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1788572548.058690:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572548.058720:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572548.059773:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360554624 00000100:00100000:3.0:1788572548.059783:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360554624 00000100:00100000:3.0:1788572548.059861:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4705 00000100:00100000:3.0:1788572548.059872:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+178:35867:x1875453360554624:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.059894:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.060166:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16471e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0537 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060245:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060260:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060271:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060284:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060290:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060295:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060623:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164733 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060633:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060641:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.060672:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa9:0x0]: returning size 3 00000004:00200000:3.0:1788572548.060676:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.060694:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060706:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164733 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060713:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060719:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060726:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060731:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.060736:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060743:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060755:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.060758:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.060763:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060770:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164725 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060782:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060788:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060794:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060800:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.060803:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060809:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060816:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x74f887a3 (ffff9626c1d8f700) 00010000:00010000:3.0:1788572548.060818:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.060820:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.060825:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16472c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.061623:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16471e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0537 expref: 180 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.061642:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74ccc00/0x85ace67feb164733 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0537 expref: 179 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.061651:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572548.061653:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.061656:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74ccc00, rc 0) 00000100:00100000:3.0:1788572548.061983:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360554624:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2111us (5449us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.061991:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4705 00000100:00100000:3.0:1788572548.062042:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382392448 00000100:00100000:3.0:1788572548.062049:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382392448 00000100:00100000:3.0:1788572548.062108:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:3.0:1788572548.062116:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382392448:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572548.062218:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382392448:12345-0@lo:1000:mdt00_005.0 Request processed in 102us (3528us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.062224:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00080000:3.0:1788572548.062239:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572548.062269:0:23058:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967663 gen 2 00000004:00080000:3.0:1788572548.062274:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967656 committed transno 4294967663 00000004:00080000:3.0:1788572548.062288:0:23058:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967657 committed transno 4294967663 00000004:00080000:3.0:1788572548.062298:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b67b80, transno 4294967660 00000004:00080000:3.0:1788572548.062307:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b66300, transno 4294967661 00000004:00080000:3.0:1788572548.062310:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b67800, transno 4294967662 00000004:00080000:3.0:1788572548.062313:0:23058:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b67480, transno 4294967663 00000100:00100000:3.0:1788572548.062318:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382392448:0@lo:1000:mdt00_005.0 00000001:00080000:3.0:1788572548.062664:0:23058:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970817 00000100:00100000:3.0:1788572548.062740:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b100 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382392576:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1788572548.062779:0:23058:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572548.062802:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d600 going to sleep for 11 seconds 00000004:00080000:2.0:1788572548.064067:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:1.0:1788572548.064132:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382392576 00000100:00100000:1.0:1788572548.064138:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382392576 00000100:00100000:1.0:1788572548.064175:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:1.0:1788572548.064192:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:23058:x1875453382392576:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1788572548.064372:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:23058:x1875453382392576:12345-0@lo:1000:mdt00_005.0 Request processed in 190us (1592us total) trans 4294967665 rc 0/0 00000100:00100000:1.0:1788572548.064379:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:2.0:1788572548.064530:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b100 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:23058:1875453382392576:0@lo:1000:mdt00_005.0 00010000:00010000:2.0:1788572548.064565:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064575:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064582:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064588:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.064593:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064600:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064616:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9627c8ed8500) 00010000:00010000:2.0:1788572548.064619:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572548.064624:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064635:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04600/0x85ace67feb164710 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064667:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064674:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064678:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064683:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.064685:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064690:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064698:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9627fb1e7400) 00010000:00010000:2.0:1788572548.064700:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572548.064702:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064707:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16000d lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064728:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064739:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064749:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064756:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064761:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064766:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064771:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.064774:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064779:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064785:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9627fb1e7400) 00010000:00010000:2.0:1788572548.064787:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.064790:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064795:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb16473a lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064803:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064809:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064815:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064819:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.064821:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064827:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064833:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9627fb1e7400) 00010000:00010000:2.0:1788572548.064836:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572548.064839:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.064845:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164709 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa4:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.065630:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:35948:x1875453360553984:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 26762us (26913us total) trans 4294970818 rc 0/0 00000100:00100000:2.0:1788572548.065641:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4704 00000020:00080000:2.0:1788572548.065646:0:23058:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1788572548.068349:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572548.068359:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572548.068405:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 167 count 101 00000040:00080000:3.0:1788572548.068412:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572548.068619:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7876680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382392704:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572548.068755:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0c12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572548.073549:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.073679:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.074203:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360555136 00000100:00100000:3.0:1788572548.074213:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360555136 00000100:00100000:3.0:1788572548.074258:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 979 00000100:00100000:3.0:1788572548.074265:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:35948:x1875453360555136:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1788572548.074626:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa3:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572548.074667:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970819 00010000:00010000:3.0:1788572548.074713:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.074730:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.074742:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.074751:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.074758:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.074764:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.074792:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572548.074887:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35948:x1875453360555136:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 620us (1207us total) trans 4294970820 rc 0/0 00000100:00100000:3.0:1788572548.074895:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 979 00000020:00080000:3.0:1788572548.074900:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788572548.075143:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572548.075150:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.075162:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572548.075166:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.075174:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.075187:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa3:0x0].0x0 (ffff9626c1d8f700) 00010000:00010000:1.0:1788572548.075191:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572548.075195:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.075201:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb164741 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.075219:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572548.075221:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572548.075223:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572548.079645:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.079694:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360555264 00000100:00100000:1.0:1788572548.079743:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360555264 00000100:00100000:1.0:1788572548.079805:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4706 00000100:00100000:1.0:1788572548.079813:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35950:x1875453360555264:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572548.079843:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa6:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572548.079962:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35950:x1875453360555264:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 149us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.079968:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4706 00000100:00100000:3.0:1788572548.080153:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7876680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382392704:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572548.080537:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360554880 00000100:00100000:2.0:1788572548.080544:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360554880 00000100:00100000:2.0:1788572548.080577:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1338 00000100:00100000:2.0:1788572548.080585:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+54:2349:x1875453360554880:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.080596:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.080618:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000edddce01/0x85ace67feb164748 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f053e expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572548.080648:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x59:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:2.0:1788572548.080664:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=896s v=(0 0) 00010000:00010000:2.0:1788572548.080668:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.080677:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000edddce01/0x85ace67feb164748 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f053e expref: 55 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.080695:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000edddce01/0x85ace67feb164748 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f053e expref: 55 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.080709:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05c00, rc 0) 00000100:00100000:2.0:1788572548.080744:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+55:2349:x1875453360554880:12345-192.168.202.40@tcp:101:diff.0 Request processed in 160us (7207us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.080751:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1338 00000100:00100000:1.0:1788572548.084228:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.084327:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360555392 00000100:00100000:3.0:1788572548.084338:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360555392 00000100:00100000:3.0:1788572548.084396:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4707 00000100:00100000:3.0:1788572548.084406:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35950:x1875453360555392:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.084499:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084518:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084532:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084548:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084557:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084565:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084592:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084601:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084609:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.084646:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa6:0x0]: returning size 3 00000004:00200000:3.0:1788572548.084651:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.084666:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084677:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084685:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084692:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084699:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.084704:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084712:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084726:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572548.084729:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.084733:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084739:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb16474f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084751:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084757:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084762:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084766:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.084768:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084773:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084780:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x62c24e7 (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572548.084782:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.084784:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084789:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164756 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x62c24e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084807:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084813:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084818:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084822:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.084824:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084830:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084837:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:3.0:1788572548.084840:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.084842:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.084848:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb16475d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.084940:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35950:x1875453360555392:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 535us (721us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.084949:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4707 00000100:00100000:1.0:1788572548.088178:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.088344:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360555648 00000100:00100000:3.0:1788572548.088352:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360555648 00000100:00100000:3.0:1788572548.088403:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4708 00000100:00100000:3.0:1788572548.088412:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+179:35867:x1875453360555648:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.088431:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.088460:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164764 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0545 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.088550:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16476b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.088562:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16476b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.088570:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16476b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.088577:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa9:0x0]: returning size 3 00010000:00010000:3.0:1788572548.088682:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164764 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0545 expref: 182 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.088696:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16476b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0545 expref: 181 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.088702:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572548.088704:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.088706:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572548.088752:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360555648:12345-192.168.202.40@tcp:101:diff.0 Request processed in 341us (583us total) trans 4294970821 rc 0/0 00000100:00100000:3.0:1788572548.088757:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4708 00000100:00100000:1.0:1788572548.093218:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.093344:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.093450:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360556032 00000100:00100000:3.0:1788572548.093460:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360556032 00000100:00100000:3.0:1788572548.093523:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 318 00000100:00100000:3.0:1788572548.093533:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:2349:x1875453360556032:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.093551:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.093575:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb16476b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0545 expref: 181 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.093594:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa9:0x0].0x0 (ffff9626c1d8f000) 00010000:00010000:3.0:1788572548.093598:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.093647:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:2349:x1875453360556032:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 116us (304us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.093654:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 318 00000100:00100000:2.0:1788572548.094513:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360555904 00000100:00100000:2.0:1788572548.094523:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360555904 00000100:00100000:3.0:1788572548.094740:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4709 00000100:00100000:3.0:1788572548.094746:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:35950:x1875453360555904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.094761:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.094774:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164779 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f054c expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.094864:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.094879:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.094890:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.094901:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa6:0x0]: returning size 3 00010000:00010000:3.0:1788572548.094999:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095034:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095038:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095042:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.095045:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095050:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095057:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:3.0:1788572548.095059:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572548.095062:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095065:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164780 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095099:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164779 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f054c expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095106:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572548.095107:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572548.095110:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164779 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f054c expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.095115:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572548.095159:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35950:x1875453360555904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 412us (1953us total) trans 4294970822 rc 301/301 00000100:00100000:3.0:1788572548.095164:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4709 00000100:00100000:3.0:1788572548.095554:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.095601:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360556160 00000100:00100000:1.0:1788572548.095605:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360556160 00002000:00100000:1.0:1788572548.095630:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x59:0x0] (0->4095) req@ffff9627e7876680 x1875453360556160/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.095637:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7876680 with x1875453360556160 ext(0->4095) 00002000:00010000:1.0:1788572548.095641:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7876680 00000100:00100000:1.0:1788572548.095660:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 298 00000100:00100000:1.0:1788572548.095668:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+55:2348:x1875453360556160:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.095815:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+55:2348:x1875453360556160:12345-192.168.202.40@tcp:3:diff.0 Request processed in 147us (260us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.095822:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 298 00002000:00100000:1.0:1788572548.095829:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x59:0x0] (0->4095) req@ffff9627e7876680 x1875453360556160/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.095838:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7876680 with x1875453360556160 ext(0->4095) 00002000:00010000:1.0:1788572548.095842:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7876680 00000100:00100000:0.0:1788572548.098935:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.099056:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.099168:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360556288 00000100:00100000:3.0:1788572548.099178:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360556288 00000100:00100000:3.0:1788572548.099243:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4710 00000100:00100000:3.0:1788572548.099253:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35950:x1875453360556288:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572548.099293:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0xa6:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572548.099317:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360556544 00000100:00100000:2.0:1788572548.099322:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360556544 00000100:00100000:2.0:1788572548.099364:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 980 00000100:00100000:2.0:1788572548.099370:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+180:35867:x1875453360556544:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572548.099397:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35950:x1875453360556288:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 146us (474us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.099404:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4710 00000100:00100000:2.0:1788572548.099676:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360556544:12345-192.168.202.40@tcp:35:diff.0 Request processed in 304us (618us total) trans 4294970823 rc 0/0 00000100:00100000:2.0:1788572548.099683:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 980 00000100:00100000:0.0:1788572548.102528:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.102593:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360556672 00000100:00100000:3.0:1788572548.102603:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360556672 00000100:00100000:3.0:1788572548.102658:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4711 00000100:00100000:3.0:1788572548.102667:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:35950:x1875453360556672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.102685:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.102704:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16478e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f055a expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.102769:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.102782:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.102793:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.102811:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.102818:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.102824:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103162:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1647a3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103172:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1647a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103179:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1647a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.103202:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572548.103205:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.103220:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1647a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103231:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103238:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103245:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103255:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.103259:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103267:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103278:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572548.103280:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.103285:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103291:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164795 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103301:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103307:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103313:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103318:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.103320:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103326:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103336:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572548.103338:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.103341:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103346:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16479c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103372:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16478e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f055a expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103383:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1647a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f055a expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.103389:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.103391:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572548.103394:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d600, rc 0) 00000100:00100000:3.0:1788572548.103459:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35950:x1875453360556672:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 792us (939us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.103466:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4711 00000100:00100000:3.0:1788572548.105544:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.105627:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360556800 00000100:00100000:1.0:1788572548.105636:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360556800 00000100:00100000:1.0:1788572548.105692:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4712 00000100:00100000:1.0:1788572548.105702:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+181:35867:x1875453360556800:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.105725:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.105744:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1647aa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0553 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.105803:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.105817:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.105827:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.105843:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.105858:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.105865:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106214:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1647bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106222:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1647bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106227:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1647bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572548.106253:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa8:0x0]: returning size 3 00000004:00200000:1.0:1788572548.106256:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572548.106270:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1647bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106279:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1647bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106287:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106291:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106295:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106298:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.106301:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106306:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106314:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572548.106316:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.106319:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106324:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb1647b1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106333:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106339:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106345:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106350:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.106353:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106358:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106365:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xbd5240a3 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572548.106368:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.106370:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106375:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb1647b8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106404:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb1647aa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0553 expref: 183 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106417:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97400/0x85ace67feb1647bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0553 expref: 182 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.106423:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:1.0:1788572548.106426:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572548.106428:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97400, rc 0) 00000100:00100000:1.0:1788572548.106599:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360556800:12345-192.168.202.40@tcp:101:diff.0 Request processed in 898us (1062us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.106605:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4712 00000100:00100000:0.0:1788572548.108353:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.108419:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360557056 00000100:00100000:3.0:1788572548.108428:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360557056 00000100:00100000:3.0:1788572548.108494:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4713 00000100:00100000:3.0:1788572548.108504:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35950:x1875453360557056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.108525:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.108527:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572548.108552:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1647a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f055a expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.108568:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.108574:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572548.108585:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1647c6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0561 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572548.109458:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5a:0x0], index 0: 0 00010000:00010000:3.0:1788572548.109643:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1647cd lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.109658:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1647cd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.109667:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1647cd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.109718:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1647c6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0561 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.109732:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1647cd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0561 expref: 83 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.109741:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9626c1d8ff00) 00010000:00010000:3.0:1788572548.109744:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.109747:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d800, rc 0) 00000100:00100000:3.0:1788572548.109873:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35950:x1875453360557056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1370us (1527us total) trans 4294970824 rc 0/0 00000100:00100000:3.0:1788572548.109886:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4713 00000100:00100000:0.0:1788572548.111727:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.111829:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360557184 00000100:00100000:2.0:1788572548.111838:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360557184 00000100:00100000:2.0:1788572548.111930:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1339 00000100:00100000:2.0:1788572548.111941:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+55:2350:x1875453360557184:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.111980:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.112025:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1647db lrc: 2/0,0 mode: --/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0568 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572548.112053:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x55:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:2.0:1788572548.112070:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=896s v=(0 0) 00010000:00010000:2.0:1788572548.112075:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.112084:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1647db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0568 expref: 56 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.112100:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000097f34723/0x85ace67feb1647db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0568 expref: 56 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.112114:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04600, rc 0) 00000100:00100000:2.0:1788572548.112170:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2350:x1875453360557184:12345-192.168.202.40@tcp:101:diff.0 Request processed in 230us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.112176:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1339 00000100:00100000:2.0:1788572548.114996:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.115094:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360557312 00000100:00100000:3.0:1788572548.115104:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360557312 00000100:00100000:3.0:1788572548.115165:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4714 00000100:00100000:3.0:1788572548.115173:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35950:x1875453360557312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.115204:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.115221:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb1647e2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f056f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115278:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115289:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115298:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115308:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115312:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115316:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115633:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1647f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115647:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1647f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115655:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1647f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.115688:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa2:0x0]: returning size 16384 00000004:00200000:3.0:1788572548.115692:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.115711:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1647f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115723:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115730:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115737:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115743:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.115748:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115756:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115770:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572548.115774:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.115779:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115791:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1647e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115805:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115811:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115818:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115823:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14c600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.115825:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115832:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115839:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x6e67fe1f (ffff9626c1d8fe00) 00010000:00010000:3.0:1788572548.115842:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.115845:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115850:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1647f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5f:0x0].0x6e67fe1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.115886:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb1647e2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f056f expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.117526:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1647f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f056f expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.117542:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.117545:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572548.117549:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572548.117618:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35950:x1875453360557312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2445us (2628us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.117626:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4714 00000100:00100000:1.0:1788572548.118040:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.118107:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360557568 00000100:00100000:2.0:1788572548.118117:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360557568 00000100:00100000:2.0:1788572548.118172:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4715 00000100:00100000:2.0:1788572548.118183:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+182:35867:x1875453360557568:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.118207:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.118231:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1647fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0576 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.118662:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164805 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.118679:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164805 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.118690:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164805 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.118708:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa8:0x0]: returning size 3 00010000:00010000:2.0:1788572548.118857:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb1647fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0576 expref: 185 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.118875:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164805 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0576 expref: 184 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.118882:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:2.0:1788572548.118885:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.118887:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00000100:00100000:2.0:1788572548.118950:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360557568:12345-192.168.202.40@tcp:101:diff.0 Request processed in 768us (909us total) trans 4294970825 rc 0/0 00000100:00100000:2.0:1788572548.118957:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4715 00000100:00100000:1.0:1788572548.121427:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.121496:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.121563:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360557824 00000100:00100000:3.0:1788572548.121573:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360557824 00002000:00100000:3.0:1788572548.121628:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x55:0x0] (0->4095) req@ffff962801f95180 x1875453360557824/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.121639:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f95180 with x1875453360557824 ext(0->4095) 00002000:00010000:3.0:1788572548.121646:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f95180 00000100:00100000:3.0:1788572548.121682:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 299 00000100:00100000:3.0:1788572548.121693:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2348:x1875453360557824:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572548.122063:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2348:x1875453360557824:12345-192.168.202.40@tcp:3:diff.0 Request processed in 370us (565us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572548.122072:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 299 00002000:00100000:3.0:1788572548.122080:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x55:0x0] (0->4095) req@ffff962801f95180 x1875453360557824/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.122088:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f95180 with x1875453360557824 ext(0->4095) 00002000:00010000:3.0:1788572548.122093:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f95180 00000100:00100000:2.0:1788572548.122456:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360557696 00000100:00100000:2.0:1788572548.122462:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360557696 00000100:00100000:2.0:1788572548.122511:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 319 00000100:00100000:2.0:1788572548.122520:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:2350:x1875453360557696:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.122533:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.122556:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164805 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0576 expref: 184 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.122572:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa8:0x0].0x0 (ffff9627c8ed8e00) 00010000:00010000:2.0:1788572548.122577:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572548.122612:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+183:2350:x1875453360557696:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 94us (1195us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.122620:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 319 00000100:00100000:0.0:1788572548.125872:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.125966:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360558080 00000100:00100000:1.0:1788572548.125976:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360558080 00000100:00100000:1.0:1788572548.126068:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4716 00000100:00100000:1.0:1788572548.126080:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35950:x1875453360558080:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572548.126139:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572548.126166:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1647cd lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0561 expref: 84 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126200:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9626c1d8ff00) 00010000:00010000:1.0:1788572548.126218:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb1647f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f056f expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126227:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572548.126231:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572548.126254:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126267:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126277:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126292:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126299:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126306:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126534:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126543:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126549:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126555:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.126559:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126566:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126575:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572548.126578:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.126581:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126586:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164813 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126595:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126601:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126607:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126612:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.126615:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126621:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126628:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x270461b6 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572548.126630:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.126632:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.126638:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16481a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa2:0x0].0x270461b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.126730:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35950:x1875453360558080:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 651us (870us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572548.126738:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4716 00000100:00100000:3.0:1788572548.128044:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.128145:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360558336 00000100:00100000:1.0:1788572548.128156:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360558336 00000100:00100000:1.0:1788572548.128207:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 981 00000100:00100000:1.0:1788572548.128217:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+183:35867:x1875453360558336:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.128655:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360558336:12345-192.168.202.40@tcp:35:diff.0 Request processed in 438us (613us total) trans 4294970826 rc 0/0 00000100:00100000:1.0:1788572548.128662:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 981 00000100:00100000:0.0:1788572548.129974:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.130476:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360558464 00000100:00100000:2.0:1788572548.130486:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360558464 00000100:00100000:2.0:1788572548.130547:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4717 00000100:00100000:2.0:1788572548.130558:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:35950:x1875453360558464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572548.130579:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.130604:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164821 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f057d expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.130641:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.130654:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.130666:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.130759:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04e00/0x85ace67feb164821 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 84 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.130776:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 83 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.130786:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572548.130789:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572548.130793:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05800, rc 0) 00000100:00100000:2.0:1788572548.130856:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35950:x1875453360558464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 299us (887us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.130864:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4717 00000100:00100000:0.0:1788572548.132122:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.132226:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360558592 00000100:00100000:1.0:1788572548.132233:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360558592 00000100:00100000:1.0:1788572548.132275:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4718 00000100:00100000:1.0:1788572548.132285:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360558592:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.132308:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.132325:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16482f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0584 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132387:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132401:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132412:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132423:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132427:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132443:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132761:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb164844 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132770:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb164844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132782:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb164844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572548.132815:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa7:0x0]: returning size 3 00000004:00200000:1.0:1788572548.132819:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572548.132836:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb164844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132980:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb164844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132990:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.132997:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133035:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133042:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.133045:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133053:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133064:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572548.133067:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.133072:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133082:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb164836 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133093:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133099:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133105:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133109:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a96c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.133111:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133116:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133123:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x5abf9a3 (ffff9627c8ed8500) 00010000:00010000:1.0:1788572548.133125:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.133127:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133132:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96c00/0x85ace67feb16483d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133162:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97a00/0x85ace67feb16482f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0584 expref: 186 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133176:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96a00/0x85ace67feb164844 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0584 expref: 185 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.133185:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627d1184000) 00010000:00010000:1.0:1788572548.133187:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572548.133190:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96a00, rc 0) 00000100:00100000:1.0:1788572548.133247:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:35867:x1875453360558592:12345-192.168.202.40@tcp:101:diff.0 Request processed in 963us (1132us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.133254:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4718 00000100:00100000:0.0:1788572548.134201:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.134267:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360558848 00000100:00100000:1.0:1788572548.134275:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360558848 00000100:00100000:1.0:1788572548.134326:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4719 00000100:00100000:1.0:1788572548.134335:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:35950:x1875453360558848:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572548.134417:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.134443:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.134453:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572548.134822:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].78, 88 off14576 00000004:00010000:1.0:1788572548.134917:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000998e28dd reply state 0000000066e06271 transno 4294970827 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.134926:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572548.135035:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35950:x1875453360558848:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 700us (838us total) trans 4294970827 rc 0/0 00000100:00100000:1.0:1788572548.135043:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4719 00010000:00010000:2.0:1788572548.137131:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137153:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137160:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137167:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.137174:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137183:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137203:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572548.137206:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572548.137211:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137217:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb16484b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.137373:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.137414:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.137420:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360558976 00000100:00100000:2.0:1788572548.137430:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360558976 00000100:00100000:3.0:1788572548.137491:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360559104 00000100:00100000:3.0:1788572548.137496:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360559104 00000100:00100000:2.0:1788572548.137554:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1340 00000100:00100000:2.0:1788572548.137566:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+43:2349:x1875453360558976:12345-192.168.202.40@tcp:101:diff.0 00000100:00100000:3.0:1788572548.137576:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 360 00000100:00100000:3.0:1788572548.137583:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:2347:x1875453360559104:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572548.137587:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.137606:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb164852 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05a0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.137611:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.137622:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164859 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f058b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572548.137631:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x57:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:2.0:1788572548.137644:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=896s v=(0 0) 00010000:00010000:2.0:1788572548.137649:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x57:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.137658:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb164852 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05a0 expref: 44 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137674:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199bc967/0x85ace67feb164852 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x57:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05a0 expref: 44 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.137674:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.137687:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04e00, rc 0) 00010000:00010000:3.0:1788572548.137688:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.137698:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.137710:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.137716:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.137722:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.137747:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+44:2349:x1875453360558976:12345-192.168.202.40@tcp:101:diff.0 Request processed in 184us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.137753:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1340 00010000:00010000:3.0:1788572548.138175:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138187:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138193:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.138216:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa6:0x0]: returning size 3 00000004:00200000:3.0:1788572548.138221:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.138242:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x53/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138252:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138259:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138262:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138267:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138270:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cd800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.138272:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138276:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138282:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.138284:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.138286:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138290:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164860 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138297:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138300:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138304:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138307:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.138309:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138312:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138315:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x4e05b2a3 (ffff9626c1d8fe00) 00010000:00010000:3.0:1788572548.138317:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.138318:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138321:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164867 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138343:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164859 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 187 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138351:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 186 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138355:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:3.0:1788572548.138357:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572548.138358:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00010000:00010000:3.0:1788572548.138363:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.138368:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164875 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0592 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138384:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138389:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138394:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138402:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138406:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138410:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138591:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16488a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138600:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16488a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138608:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16488a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.138622:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa5:0x0]: returning size 3 00000004:00200000:3.0:1788572548.138625:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.138635:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16488a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138643:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16488a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138649:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138654:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138660:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138665:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.138668:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138674:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138683:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.138686:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.138689:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138695:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb16487c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138704:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138709:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138715:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138720:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.138722:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138728:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138734:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x965f6ba3 (ffff9626c1d8f200) 00010000:00010000:3.0:1788572548.138737:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.138740:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138745:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164883 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138767:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164875 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0592 expref: 188 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138779:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14ce00/0x85ace67feb16488a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0592 expref: 187 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138791:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9626c1d8f600) 00010000:00010000:3.0:1788572548.138793:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.138796:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14ce00, rc 0) 00010000:00010000:3.0:1788572548.138804:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.138812:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb164891 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0599 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138836:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138844:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138852:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138862:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138869:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.138876:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139382:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1648a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139394:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1648a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139401:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1648a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.139417:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa4:0x0]: returning size 3 00000004:00200000:3.0:1788572548.139420:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.139445:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1648a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139453:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1648a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139460:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139466:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139472:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139476:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14cc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.139479:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139485:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139494:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.139496:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.139500:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139504:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164898 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139514:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139519:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139523:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139527:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.139530:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139534:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139539:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xdeb924a3 (ffff9626c1d8fd00) 00010000:00010000:3.0:1788572548.139541:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.139544:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139549:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb16489f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139575:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb164891 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0599 expref: 189 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139586:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d600/0x85ace67feb1648a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0599 expref: 188 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139592:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9626c1d8f300) 00010000:00010000:3.0:1788572548.139594:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.139596:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d600, rc 0) 00010000:00010000:3.0:1788572548.139602:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.139613:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb1648ad lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05a7 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139639:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139647:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139653:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139663:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139667:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139671:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139739:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139745:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139748:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139751:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.139753:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139756:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139762:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.139764:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.139766:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139770:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb1648b4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139776:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139779:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139783:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139786:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3336800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.139788:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139792:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139796:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xc8545ca3 (ffff9626c1d8fa00) 00010000:00010000:3.0:1788572548.139797:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.139799:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139803:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb1648bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139824:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb1648ad lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05a7 expref: 189 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139834:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.139835:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572548.139838:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb1648ad lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05a7 expref: 189 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.139847:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3337a00, rc -2) 00000100:00100000:3.0:1788572548.141807:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:2347:x1875453360559104:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 4209us (4377us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572548.141823:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 360 00000100:00100000:0.0:1788572548.143345:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.143687:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360559232 00000100:00100000:3.0:1788572548.143701:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360559232 00000100:00100000:3.0:1788572548.143766:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4720 00000100:00100000:3.0:1788572548.143777:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:35950:x1875453360559232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.143807:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.143834:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05c3 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.143916:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.143934:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 85 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.143956:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572548.143959:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572548.143964:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3336400, rc 0) 00000100:00100000:3.0:1788572548.144039:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35950:x1875453360559232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 263us (704us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.144046:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4720 00000100:00100000:0.0:1788572548.144887:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.145515:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360559488 00000100:00100000:3.0:1788572548.145523:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360559488 00000100:00100000:3.0:1788572548.145566:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4721 00000100:00100000:3.0:1788572548.145575:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360559488:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.145591:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.145611:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb1648c9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05ca expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.145715:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648d0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.145735:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.145748:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.145760:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa7:0x0]: returning size 3 00010000:00010000:3.0:1788572548.145923:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb1648c9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05ca expref: 191 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.145948:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648d0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05ca expref: 190 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.145958:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627d1184000) 00010000:00010000:3.0:1788572548.145960:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.145965:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c37d7c00, rc 0) 00000100:00100000:3.0:1788572548.146054:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360559488:12345-192.168.202.40@tcp:101:diff.0 Request processed in 478us (1169us total) trans 4294970828 rc 0/0 00000100:00100000:3.0:1788572548.146063:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4721 00000100:00100000:1.0:1788572548.149730:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.150427:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360559616 00000100:00100000:3.0:1788572548.150452:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360559616 00000100:00100000:3.0:1788572548.150512:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1341 00000100:00100000:3.0:1788572548.150523:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360559616:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1788572548.150840:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360559616:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 318us (1120us total) trans 4294967666 rc 0/0 00000100:00100000:3.0:1788572548.150848:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1341 00000100:00100000:0.0:1788572548.154242:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.154287:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.154402:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360559744 00000100:00100000:1.0:1788572548.154415:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360559744 00000100:00100000:1.0:1788572548.154482:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360559872 00000100:00100000:1.0:1788572548.154486:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360559872 00000100:00100000:1.0:1788572548.154525:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1342 00000100:00100000:1.0:1788572548.154538:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2348:x1875453360559744:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:1.0:1788572548.154561:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.154592:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000058b77b55/0x85ace67feb1648de lrc: 2/0,0 mode: --/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05d1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572548.154624:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x56:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:1.0:1788572548.154650:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=896s v=(0 0) 00010000:00010000:1.0:1788572548.154655:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x56:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572548.154666:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000058b77b55/0x85ace67feb1648de lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05d1 expref: 57 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.154688:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000058b77b55/0x85ace67feb1648de lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x56:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05d1 expref: 57 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.154705:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96c00, rc 0) 00000100:00100000:1.0:1788572548.154780:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2348:x1875453360559744:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 245us (548us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.154789:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1342 00000100:00100000:1.0:1788572548.154809:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1343 00000100:00100000:1.0:1788572548.154818:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+44:2348:x1875453360559872:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:1.0:1788572548.154825:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.154838:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb1648e5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05d8 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572548.154856:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x59:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:1.0:1788572548.154867:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=896s v=(0 0) 00010000:00010000:1.0:1788572548.154870:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x59:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572548.154875:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb1648e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05d8 expref: 45 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.154887:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f46fa957/0x85ace67feb1648e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x59:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05d8 expref: 45 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.154896:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97a00, rc 0) 00000100:00100000:1.0:1788572548.154929:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+45:2348:x1875453360559872:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 113us (642us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.154935:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1343 00000100:00100000:2.0:1788572548.156608:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.156655:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.156720:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360560128 00000100:00100000:0.0:1788572548.156728:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360560128 00000100:00100000:0.0:1788572548.156793:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 320 00000100:00100000:0.0:1788572548.156808:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:2348:x1875453360560128:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572548.156825:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572548.156849:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05ca expref: 190 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.156867:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa7:0x0].0x0 (ffff9627d1184000) 00010000:00010000:0.0:1788572548.156875:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572548.156949:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+189:2348:x1875453360560128:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 148us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.156957:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 320 00000100:00100000:1.0:1788572548.157613:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360560000 00000100:00100000:1.0:1788572548.157619:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360560000 00002000:00100000:1.0:1788572548.157654:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x57:0x0] (0->4095) req@ffff9626c1bcd880 x1875453360560000/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.157662:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bcd880 with x1875453360560000 ext(0->4095) 00002000:00010000:1.0:1788572548.157666:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1bcd880 00000100:00100000:1.0:1788572548.157701:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 300 00000100:00100000:1.0:1788572548.157709:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+45:2349:x1875453360560000:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.158267:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+45:2349:x1875453360560000:12345-192.168.202.40@tcp:3:diff.0 Request processed in 556us (1671us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.158277:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 300 00002000:00100000:1.0:1788572548.158285:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x57:0x0] (0->4095) req@ffff9626c1bcd880 x1875453360560000/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.158293:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1bcd880 with x1875453360560000 ext(0->4095) 00002000:00010000:1.0:1788572548.158297:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1bcd880 00000100:00100000:2.0:1788572548.160651:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.161394:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360560384 00000100:00100000:3.0:1788572548.161411:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360560384 00000100:00100000:3.0:1788572548.161497:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 982 00000100:00100000:3.0:1788572548.161509:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:35950:x1875453360560384:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572548.161940:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35950:x1875453360560384:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 432us (1297us total) trans 4294970829 rc 0/0 00000100:00100000:3.0:1788572548.161948:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 982 00000100:00100000:2.0:1788572548.162860:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.163236:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360560640 00000100:00100000:3.0:1788572548.163243:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360560640 00000100:00100000:3.0:1788572548.163495:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 983 00000100:00100000:3.0:1788572548.163506:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+189:35867:x1875453360560640:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572548.163813:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360560640:12345-192.168.202.40@tcp:35:diff.0 Request processed in 308us (956us total) trans 4294970830 rc 0/0 00000100:00100000:3.0:1788572548.163821:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 983 00000100:00100000:2.0:1788572548.164774:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.164820:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360560896 00000100:00100000:1.0:1788572548.164828:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360560896 00000100:00100000:1.0:1788572548.164937:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4722 00000100:00100000:1.0:1788572548.164948:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35950:x1875453360560896:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572548.165058:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35950:x1875453360560896:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 111us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.165065:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4722 00000100:00100000:2.0:1788572548.167617:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.167737:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360561024 00000100:00100000:3.0:1788572548.167749:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360561024 00000100:00100000:3.0:1788572548.167863:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4723 00000100:00100000:3.0:1788572548.167873:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:35950:x1875453360561024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.167892:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.167914:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb1648ec lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05df expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.167989:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1648f3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.168025:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1648f3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.168039:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1648f3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.168077:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.168110:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1648f3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.168140:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb1648ec lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05df expref: 88 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.168159:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1648f3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05df expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.168170:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572548.168173:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1788572548.168177:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c37d7000, rc 0) 00000100:00100000:3.0:1788572548.168228:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35950:x1875453360561024:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 357us (619us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.168236:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4723 00000100:00100000:2.0:1788572548.171650:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.171694:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.171771:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360561280 00000100:00100000:3.0:1788572548.171780:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360561280 00000100:00100000:3.0:1788572548.171887:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4724 00000100:00100000:3.0:1788572548.171899:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360561280:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.171929:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.171953:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648fa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05e6 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172082:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172110:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172123:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.172133:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa6:0x0]: returning size 3 00010000:00010000:3.0:1788572548.172229:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172236:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172243:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172249:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.172253:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172261:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172284:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:3.0:1788572548.172287:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572548.172291:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172303:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337e00/0x85ace67feb164901 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172344:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648fa lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05e6 expref: 192 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172353:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572548.172355:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572548.172359:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb1648fa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05e6 expref: 192 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.172368:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c37d7c00, rc 0) 00000100:00100000:3.0:1788572548.172424:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:35867:x1875453360561280:12345-192.168.202.40@tcp:101:diff.0 Request processed in 528us (783us total) trans 4294970831 rc 301/301 00000100:00100000:3.0:1788572548.172444:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4724 00000100:00100000:0.0:1788572548.172513:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360561408 00000100:00100000:0.0:1788572548.172518:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360561408 00000100:00100000:0.0:1788572548.172551:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 361 00000100:00100000:0.0:1788572548.172558:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:2349:x1875453360561408:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572548.172608:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.172618:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb16490f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f05ed expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172651:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172660:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172669:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172684:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172691:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172698:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172894:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172903:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172910:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172915:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23eee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.172919:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172925:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172932:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572548.172935:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.172938:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172943:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164916 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172952:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172959:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172965:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172969:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.172973:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172979:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172985:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x2f385ca3 (ffff9627fb1e7d00) 00010000:00010000:0.0:1788572548.172987:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.172990:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.172995:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16491d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.173050:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb16490f lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05ed expref: 192 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.173058:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572548.173060:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572548.173064:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb16490f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05ed expref: 192 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.173071:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc -2) 00000100:00100000:0.0:1788572548.173111:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:2349:x1875453360561408:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 554us (1416us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572548.173117:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 361 00000100:00100000:2.0:1788572548.175788:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.176558:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360561536 00000100:00100000:3.0:1788572548.176572:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360561536 00000100:00100000:3.0:1788572548.176661:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1344 00000100:00100000:3.0:1788572548.176675:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7660450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360561536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.176700:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.176722:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f062c expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.176752:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6a9b7383, blocks: 0x1 00010000:00010000:3.0:1788572548.176773:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.176776:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.176785:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.176799:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.176809:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3337a00, rc 0) 00000100:00100000:3.0:1788572548.176874:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7660450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360561536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 203us (1097us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.176882:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1344 00000100:00100000:2.0:1788572548.179239:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.179323:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.179516:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360561920 00000100:00100000:1.0:1788572548.179526:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360561920 00000100:00100000:1.0:1788572548.179588:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360561664 00000100:00100000:1.0:1788572548.179591:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360561664 00000100:00100000:1.0:1788572548.179623:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4725 00000100:00100000:1.0:1788572548.179633:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35950:x1875453360561920:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572548.179749:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35950:x1875453360561920:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 117us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.179756:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4725 00000100:00100000:1.0:1788572548.179774:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4726 00000100:00100000:1.0:1788572548.179778:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:35867:x1875453360561664:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.179792:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.179811:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0633 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.179864:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 2/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.179874:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 4/0,0 mode: --/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 192 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.179894:0:5839:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.179920:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 3/0,0 mode: --/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 192 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.179935:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:1.0:1788572548.179937:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572548.179940:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 3/0,0 mode: --/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 192 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.179946:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a96000, rc 0) 00000100:00100000:1.0:1788572548.179962:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+192:35867:x1875453360561664:12345-192.168.202.40@tcp:101:diff.0 Request processed in 183us (640us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.179965:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4726 00000100:00100000:1.0:1788572548.182064:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.182311:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562048 00000100:00100000:2.0:1788572548.182322:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562048 00000100:00100000:2.0:1788572548.182374:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1345 00000100:00100000:2.0:1788572548.182383:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:35950:x1875453360562048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572548.182397:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.182415:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f063a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.182454:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.182462:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.182466:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.182470:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.182503:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.182512:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 13 pid: 20911 timeout: 996 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.182524:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382392832:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572548.182636:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b83c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572548.185219:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.185308:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562176 00000100:00100000:3.0:1788572548.185317:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562176 00010000:00010000:3.0:1788572548.185353:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 14 pid: 20911 timeout: 996 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.185384:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1120 00000100:00100000:3.0:1788572548.185392:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efc050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360562176:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.185407:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.185443:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 14 pid: 20911 timeout: 996 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.185457:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 14 pid: 20911 timeout: 996 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.185497:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.185503:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.185515:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.185527:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.185577:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 14 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.185593:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7876d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382392960:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572548.185758:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5dd6780 going to sleep for 11 seconds 00010000:00010000:1.0:1788572548.185909:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a1dcbe9/0x85ace67feb164924 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f062c expref: 14 pid: 20911 timeout: 996 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.185930:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382392832:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572548.185959:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 13 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.185971:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:1.0:1788572548.186044:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:35950:x1875453360562048:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3621us (3948us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.186052:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1345 00000100:00100000:0.0:1788572548.189047:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7876d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:23057:1875453382392960:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572548.189085:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572548.189168:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efc050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360562176:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3773us (3954us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.189183:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1120 00000100:00100000:0.0:1788572548.192193:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.192234:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562304 00000100:00100000:1.0:1788572548.192248:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562304 00002000:00100000:1.0:1788572548.192301:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5a:0x0] (0->2) req@ffff9626c12eb480 x1875453360562304/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:209/0 lens 496/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.192310:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12eb480 with x1875453360562304 ext(0->2) 00002000:00010000:1.0:1788572548.192315:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c12eb480 00000100:00100000:1.0:1788572548.192346:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 301 00000100:00100000:1.0:1788572548.192353:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360562304:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1788572548.192917:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360562304:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 560us (734us total) trans 4294967667 rc 0/0 00000100:00100000:1.0:1788572548.192927:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 301 00002000:00100000:1.0:1788572548.192937:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5a:0x0] (0->2) req@ffff9626c12eb480 x1875453360562304/t4294967667(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:209/0 lens 496/448 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.192947:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12eb480 with x1875453360562304 ext(0->2) 00002000:00010000:1.0:1788572548.192951:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c12eb480 00000100:00100000:2.0:1788572548.195209:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.195296:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562432 00000100:00100000:0.0:1788572548.195304:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562432 00000100:00100000:0.0:1788572548.195357:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4727 00000100:00100000:0.0:1788572548.195367:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:35950:x1875453360562432:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1788572548.213428:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970817 is committed 00000001:00080000:2.0:1788572548.213493:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970827 is committed 00000001:00080000:2.0:1788572548.213500:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970826 is committed 00000020:00080000:2.0:1788572548.213515:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ec077300 refcount 2 committed 0 result 0 batchid 143 00000020:00080000:2.0:1788572548.213520:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c687cd00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c7e79540 00000020:00080000:2.0:1788572548.213524:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c687c600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f5976400 00000020:00080000:2.0:1788572548.213561:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 141 batchid updated 00000001:00080000:2.0:1788572548.213568:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970831 is committed 00000001:00080000:2.0:1788572548.213576:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970829 is committed 00000100:00100000:0.0:1788572548.213579:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:35950:x1875453360562432:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 18207us (18371us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.213591:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4727 00000020:00080000:2.0:1788572548.213594:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627fb17dc00 refcount 2 committed 0 result 0 batchid 144 00000020:00080000:2.0:1788572548.213597:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc900 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627fb17d840 00000020:00080000:2.0:1788572548.213600:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627f5976000 00000020:00080000:2.0:1788572548.213606:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627fb17d000 refcount 2 committed 0 result 0 batchid 142 00000020:00080000:2.0:1788572548.213608:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcc400 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627fb17dcc0 00000020:00080000:2.0:1788572548.213610:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801fcce00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c10abc00 00000020:00080000:1.0:1788572548.216077:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 144 committed batchid 141 00000100:00100000:2.0:1788572548.216086:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.216290:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562560 00000100:00100000:3.0:1788572548.216301:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562560 00000100:00100000:3.0:1788572548.216361:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1346 00000100:00100000:3.0:1788572548.216374:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2348:x1875453360562560:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000020:00080000:1.0:1788572548.216610:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 141 00010000:00010000:0.0:1788572548.218171:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1788572548.233903:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967664 is committed 00000001:00080000:0.0:1788572548.233920:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967667 is committed 00000001:00080000:0.0:1788572548.233928:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967665 is committed 00000100:00100000:2.0:1788572548.234385:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360562560:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 18007us (18297us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.234401:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1346 00000100:00100000:0.0:1788572548.238218:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.238377:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562816 00000100:00100000:1.0:1788572548.238386:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562816 00000100:00100000:1.0:1788572548.238450:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4728 00000100:00100000:1.0:1788572548.238461:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:35950:x1875453360562816:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572548.238524:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572548.238552:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1648f3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05df expref: 59 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.238569:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:1.0:1788572548.238573:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572548.238603:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.238613:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.238620:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572548.238770:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008db002e5 reply state 00000000d7ae8fb8 transno 4294970832 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.238777:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572548.239626:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:35950:x1875453360562816:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1163us (1414us total) trans 4294970832 rc 0/0 00000100:00100000:0.0:1788572548.239633:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4728 00010000:00010000:3.0:1788572548.241406:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.241420:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.241425:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.241428:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7a97600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.241441:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.241450:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.241470:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572548.241473:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.241479:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.241487:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97600/0x85ace67feb164939 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.243505:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.243725:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360562944 00000100:00100000:3.0:1788572548.243741:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360562944 00000100:00100000:3.0:1788572548.243823:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1347 00000100:00100000:3.0:1788572548.243836:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360562944:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1788572548.244199:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb size from disk: 0 -> 3 00002000:00010000:3.0:1788572548.244202:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb mtime from disk: 0 -> 1788572517 00002000:00010000:3.0:1788572548.244205:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb atime from disk: 0 -> 1788572517 00002000:00010000:3.0:1788572548.244207:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb ctime from disk: 1788572547 -> 1788572526 00002000:00010000:3.0:1788572548.244209:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1788572548.244284:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360562944:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 450us (797us total) trans 4294967668 rc 0/0 00000100:00100000:3.0:1788572548.244292:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1347 00000100:00100000:0.0:1788572548.247147:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.247246:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360563200 00000100:00100000:2.0:1788572548.247258:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360563200 00000100:00100000:2.0:1788572548.247321:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4729 00000100:00100000:2.0:1788572548.247332:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:35950:x1875453360563200:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572548.247405:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.247422:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 59 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.247440:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.247489:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 59 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.247506:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 59 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.247521:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382393088:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572548.247603:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8540 going to sleep for 11 seconds 00000100:00100000:2.0:1788572548.249813:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.250141:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360563328 00000100:00100000:1.0:1788572548.250150:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360563328 00010000:00010000:1.0:1788572548.250196:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 60 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572548.250232:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1121 00000100:00100000:1.0:1788572548.250242:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2349:x1875453360563328:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572548.250257:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572548.250264:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 60 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.250277:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 60 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.250301:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:1.0:1788572548.250306:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.250313:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.250320:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.250336:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572548.250382:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2349:x1875453360563328:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 142us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.250388:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1121 00010000:00010000:3.0:1788572548.251568:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164828 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f057d expref: 59 pid: 23058 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.251585:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382393088:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572548.251604:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251693:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251701:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251734:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251742:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251748:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251754:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.251759:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251767:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251781:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572548.251784:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572548.251788:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.251794:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164940 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.251913:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:35950:x1875453360563200:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 4573us (4770us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572548.251922:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4729 00000100:00100000:2.0:1788572548.255442:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.255546:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360563456 00000100:00100000:0.0:1788572548.255557:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360563456 00000100:00100000:0.0:1788572548.255656:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1348 00000100:00100000:0.0:1788572548.255667:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360563456:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572548.255711:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.255738:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.255746:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572548.255750:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572548.255755:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.255794:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.255807:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 11 pid: 6682 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.255823:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382393216:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572548.255917:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982dd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572548.258316:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.258416:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382393216:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572548.258463:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.258534:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360563584 00000100:00100000:3.0:1788572548.258542:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360563584 00010000:00010000:3.0:1788572548.258579:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 12 pid: 6682 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.258617:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1122 00000100:00100000:3.0:1788572548.258627:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360563584:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.258642:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.258667:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 12 pid: 6682 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.258681:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 12 pid: 6682 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.258703:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a68d18f4/0x85ace67feb164932 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f063a expref: 12 pid: 6682 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.258719:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572548.258723:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572548.258729:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.258740:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.258780:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.258849:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360563584:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 222us (542us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.258858:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1122 00010000:00010000:0.0:1788572548.259612:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259626:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259658:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259674:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259683:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259690:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572548.259695:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259704:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259720:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.259727:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d2456d5a/0x85ace67feb164947 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.259802:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360563456:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 4136us (4383us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.259811:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1348 00000100:00100000:0.0:1788572548.264269:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.264368:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360563840 00000100:00100000:3.0:1788572548.264378:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360563840 00000100:00100000:3.0:1788572548.264513:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4730 00000100:00100000:3.0:1788572548.264524:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:35950:x1875453360563840:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572548.264619:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.264639:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.264652:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.264924:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.264936:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 59 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.264944:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.264981:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 59 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.265037:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 59 pid: 5838 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.265055:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7876d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382393344:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572548.265134:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9bd2c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572548.272171:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.272323:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7876d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382393344:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572548.272352:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.272475:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360563968 00000100:00100000:3.0:1788572548.272482:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360563968 00010000:00010000:3.0:1788572548.272519:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 60 pid: 5838 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572548.272563:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1123 00000100:00100000:3.0:1788572548.272573:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2349:x1875453360563968:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.272592:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.272599:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 60 pid: 5838 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.272611:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 60 pid: 5838 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.272633:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1648c2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f05c3 expref: 60 pid: 5838 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.272654:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572548.272658:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.272669:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.272677:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.272704:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.272755:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:2349:x1875453360563968:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 183us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.272762:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1123 00010000:00010000:2.0:1788572548.272983:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.272992:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572548.273231:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970833 00000001:00080000:2.0:1788572548.273377:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970834 00000001:00080000:2.0:1788572548.273425:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970835 00000100:00100000:2.0:1788572548.273521:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5edf80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382393472:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572548.273561:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572548.273583:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788572548.274449:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382393472 00000100:00100000:3.0:1788572548.274460:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382393472 00000100:00100000:3.0:1788572548.274510:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1788572548.274520:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382393472:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572548.274592:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382393472:12345-0@lo:1000:mdt00_000.0 Request processed in 74us (1030us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.274596:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1788572548.274787:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967665 gen 2 00000004:00080000:2.0:1788572548.274793:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967665 committed transno 4294967665 00000100:00100000:2.0:1788572548.274815:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5edf80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382393472:0@lo:1000:mdt00_000.0 00000001:00080000:2.0:1788572548.275337:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970836 00000100:00100000:2.0:1788572548.275412:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5edf80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382393600:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572548.275918:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572548.275984:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b86c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572548.276133:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382393600 00000100:00100000:0.0:1788572548.276141:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382393600 00000100:00100000:0.0:1788572548.276278:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1788572548.276289:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382393600:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572548.276502:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382393600:12345-0@lo:1000:mdt00_000.0 Request processed in 214us (584us total) trans 4294967669 rc 0/0 00000100:00100000:0.0:1788572548.276511:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:2.0:1788572548.276576:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5edf80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382393600:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1788572548.276616:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276626:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276635:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276642:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.276647:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276656:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276673:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572548.276676:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572548.276680:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276687:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb164955 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276720:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276727:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276732:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276737:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.276740:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276750:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276758:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:2.0:1788572548.276760:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572548.276768:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 183 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276778:0:5837:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 183 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276788:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 183 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.276818:0:5837:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a96000/0x85ace67feb16492b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0633 expref: 183 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.276831:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0df80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382393728:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572548.276901:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b86c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572548.281505:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.281664:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0df80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382393728:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572548.281695:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.281706:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb16016b lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.281740:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.281754:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 183 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.281763:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 4/0,1 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.281805:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 183 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.281820:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 183 pid: 5843 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.281834:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382393856:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572548.281852:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360564096 00000100:00100000:1.0:1788572548.281858:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360564096 00000100:00100000:0.0:1788572548.281916:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982db40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572548.281923:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 984 00000100:00100000:1.0:1788572548.281933:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360564096:12345-192.168.202.40@tcp:49:diff.0 00000100:00100000:1.0:1788572548.282057:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360564096:12345-192.168.202.40@tcp:49:diff.0 Request processed in 126us (562us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.282065:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 984 00000100:00100000:0.0:1788572548.284387:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382393856:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572548.284418:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572548.287828:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.287875:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.287927:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360564224 00000100:00100000:0.0:1788572548.287941:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360564224 00000100:00100000:3.0:1788572548.287969:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360564352 00000100:00100000:3.0:1788572548.287973:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360564352 00010000:00010000:0.0:1788572548.287986:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 184 pid: 5843 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.288038:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1124 00000100:00100000:3.0:1788572548.288038:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1349 00000100:00100000:0.0:1788572548.288048:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:2349:x1875453360564224:12345-192.168.202.40@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1788572548.288049:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+45:35867:x1875453360564352:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.288064:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.288064:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.288071:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 184 pid: 5843 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.288083:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002bb9ecdb/0x85ace67feb164963 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0641 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.288084:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 184 pid: 5843 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.288105:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc200/0x85ace67feb16486e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa6:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f058b expref: 184 pid: 5843 timeout: 996 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.288119:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00002000:00010000:3.0:1788572548.288121:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5a:0x0] initial LVB size: 3, mtime: 0x6a9b7365, atime: 0x6a9b7365, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:0.0:1788572548.288125:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.288136:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.288139:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.288144:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572548.288145:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.288154:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002bb9ecdb/0x85ace67feb164963 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0641 expref: 46 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.288168:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002bb9ecdb/0x85ace67feb164963 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0641 expref: 46 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.288170:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572548.288178:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3337e00, rc 0) 00000100:00100000:3.0:1788572548.288224:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:35867:x1875453360564352:12345-192.168.202.40@tcp:101:diff.0 Request processed in 178us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.288232:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1349 00000100:00100000:0.0:1788572548.291863:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+183:2349:x1875453360564224:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3789us (4020us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.291880:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1124 00010000:00010000:0.0:1788572548.291938:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.291949:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.291964:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.291971:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.291978:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.291984:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.291989:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.291998:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292032:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:0.0:1788572548.292035:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.292040:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292052:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb16495c lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292069:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292076:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292083:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292088:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3336800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.292091:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292097:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292105:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa6:0x0].0x0 (ffff9627d55e6e00) 00010000:00010000:0.0:1788572548.292107:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572548.292110:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.292116:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16494e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572548.292424:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35950:x1875453360563840:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 27901us (28166us total) trans 4294970837 rc 0/0 00000100:00100000:0.0:1788572548.292445:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4730 00000020:00080000:0.0:1788572548.292450:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1788572548.292951:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572548.292958:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:2.0:1788572548.292962:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000040:00080000:3.0:1788572548.294797:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 101 00000040:00080000:3.0:1788572548.294818:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1788572548.295627:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360564480 00000100:00100000:1.0:1788572548.295641:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360564480 00000100:00100000:2.0:1788572548.297118:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.297182:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360564736 00000100:00100000:3.0:1788572548.297190:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360564736 00000100:00100000:3.0:1788572548.297237:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 985 00000100:00100000:3.0:1788572548.297247:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:35950:x1875453360564736:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00002000:00100000:0.0:1788572548.297821:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5a:0x0] (0->4095) req@ffff9627ffc0dc00 x1875453360564480/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.297835:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0dc00 with x1875453360564480 ext(0->4095) 00002000:00010000:0.0:1788572548.297841:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ffc0dc00 00000100:00100000:0.0:1788572548.298048:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 302 00000100:00100000:0.0:1788572548.298057:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2350:x1875453360564480:12345-192.168.202.40@tcp:3:diff.0 00000004:00080000:3.0:1788572548.298204:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa5:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000100:00100000:0.0:1788572548.298268:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2350:x1875453360564480:12345-192.168.202.40@tcp:3:diff.0 Request processed in 211us (5420us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572548.298278:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 302 00002000:00100000:0.0:1788572548.298285:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5a:0x0] (0->4095) req@ffff9627ffc0dc00 x1875453360564480/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.298293:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0dc00 with x1875453360564480 ext(0->4095) 00002000:00010000:0.0:1788572548.298297:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ffc0dc00 00000001:00080000:3.0:1788572548.298312:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970838 00010000:00010000:3.0:1788572548.298365:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298387:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298399:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298407:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298412:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298418:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298449:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572548.298517:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:35950:x1875453360564736:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1271us (1407us total) trans 4294970839 rc 0/0 00000100:00100000:3.0:1788572548.298526:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 985 00000020:00080000:3.0:1788572548.298529:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572548.298549:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572548.298555:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298563:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572548.298568:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298577:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298594:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa5:0x0].0x0 (ffff9626c1d8f900) 00010000:00010000:3.0:1788572548.298596:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572548.298601:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298607:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb16496a lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.298623:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572548.298625:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572548.298627:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572548.303597:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.304682:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360564992 00000100:00100000:1.0:1788572548.304692:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360564992 00000100:00100000:1.0:1788572548.304746:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 986 00000100:00100000:1.0:1788572548.304756:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+183:35867:x1875453360564992:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.305322:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360564992:12345-192.168.202.40@tcp:35:diff.0 Request processed in 565us (1732us total) trans 4294970840 rc 0/0 00000100:00100000:1.0:1788572548.305330:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 986 00000100:00100000:1.0:1788572548.309902:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.310223:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360565248 00000100:00100000:0.0:1788572548.310234:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360565248 00000100:00100000:0.0:1788572548.310296:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4731 00000100:00100000:0.0:1788572548.310307:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+184:35867:x1875453360565248:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.310333:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.310354:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164971 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0648 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.310476:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164978 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.310492:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164978 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.310506:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164978 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572548.310519:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa5:0x0]: returning size 3 00010000:00010000:0.0:1788572548.310656:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164971 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0648 expref: 187 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.310678:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164978 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0648 expref: 186 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.310687:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9626c1d8f600) 00010000:00010000:0.0:1788572548.310689:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572548.310693:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0030c00, rc 0) 00000100:00100000:0.0:1788572548.310765:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:35867:x1875453360565248:12345-192.168.202.40@tcp:101:diff.0 Request processed in 460us (875us total) trans 4294970841 rc 0/0 00000100:00100000:0.0:1788572548.310772:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4731 00000100:00100000:2.0:1788572548.315275:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.315322:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.315360:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360565376 00000100:00100000:1.0:1788572548.315372:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360565376 00000100:00100000:0.0:1788572548.315416:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360565504 00000100:00100000:0.0:1788572548.315422:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360565504 00002000:00100000:1.0:1788572548.315425:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x56:0x0] (0->4095) req@ffff9627ffc0c700 x1875453360565376/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.315446:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0c700 with x1875453360565376 ext(0->4095) 00002000:00010000:1.0:1788572548.315451:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ffc0c700 00000100:00100000:1.0:1788572548.315471:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 303 00000100:00100000:1.0:1788572548.315482:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2350:x1875453360565376:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572548.315514:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 321 00000100:00100000:0.0:1788572548.315522:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:2348:x1875453360565504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572548.315533:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572548.315556:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164978 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0648 expref: 186 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.315573:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa5:0x0].0x0 (ffff9626c1d8f600) 00010000:00010000:0.0:1788572548.315579:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572548.315635:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:2348:x1875453360565504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 115us (313us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.315643:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 321 00000100:00100000:1.0:1788572548.315692:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2350:x1875453360565376:12345-192.168.202.40@tcp:3:diff.0 Request processed in 211us (428us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.315700:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 303 00002000:00100000:1.0:1788572548.315705:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x56:0x0] (0->4095) req@ffff9627ffc0c700 x1875453360565376/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.315713:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0c700 with x1875453360565376 ext(0->4095) 00002000:00010000:1.0:1788572548.315716:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ffc0c700 00000100:00100000:2.0:1788572548.320938:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.323896:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360565632 00000100:00100000:0.0:1788572548.323921:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360565632 00000100:00100000:0.0:1788572548.324047:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 364 00000100:00100000:0.0:1788572548.324060:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+185:2350:x1875453360565632:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:1.0:1788572548.325756:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.326045:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360565888 00000100:00100000:2.0:1788572548.326057:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360565888 00000100:00100000:2.0:1788572548.326108:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 987 00000100:00100000:2.0:1788572548.326118:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+187:35867:x1875453360565888:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572548.326580:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360565888:12345-192.168.202.40@tcp:35:diff.0 Request processed in 461us (830us total) trans 4294970842 rc 0/0 00000100:00100000:2.0:1788572548.326588:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 987 00010000:00010000:0.0:1788572548.332440:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.332470:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164986 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f064f expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332538:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332554:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332567:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332584:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332593:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332603:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332799:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332811:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332818:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332823:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.332827:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332834:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332846:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572548.332849:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.332854:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332860:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb16498d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332870:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332876:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332882:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332887:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.332890:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332895:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332978:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x1dd915a3 (ffff9627fb1e7d00) 00010000:00010000:0.0:1788572548.332981:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.332985:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.332991:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164994 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.333066:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164986 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f064f expref: 188 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.333077:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:0.0:1788572548.333079:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572548.333083:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164986 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f064f expref: 188 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.333093:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0030e00, rc -2) 00000100:00100000:0.0:1788572548.333267:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:2350:x1875453360565632:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 9209us (12337us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572548.333277:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 364 00000100:00100000:3.0:1788572548.334068:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.334257:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.334288:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360566144 00000100:00100000:3.0:1788572548.334298:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360566144 00000100:00100000:3.0:1788572548.334362:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4732 00000100:00100000:3.0:1788572548.334372:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+186:35867:x1875453360566144:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.334391:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.334407:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16499b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f06c6 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.334515:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1649a2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.334529:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1649a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.334539:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1649a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.334555:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa4:0x0]: returning size 3 00010000:00010000:3.0:1788572548.334680:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb16499b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f06c6 expref: 189 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.334697:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1649a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f06c6 expref: 188 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.334704:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9626c1d8f300) 00010000:00010000:3.0:1788572548.334707:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.334710:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3336400, rc 0) 00000100:00100000:3.0:1788572548.334750:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360566144:12345-192.168.202.40@tcp:101:diff.0 Request processed in 380us (493us total) trans 4294970843 rc 0/0 00000100:00100000:3.0:1788572548.334757:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4732 00000100:00100000:1.0:1788572548.334818:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360566272 00000100:00100000:1.0:1788572548.334823:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360566272 00000100:00100000:1.0:1788572548.334868:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 365 00000100:00100000:1.0:1788572548.334875:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:2350:x1875453360566272:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572548.342468:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.342501:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb1649b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f06cd expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342566:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342582:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342595:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342609:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342616:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342623:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342824:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342833:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.342840:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343214:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23eee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.343220:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343229:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343242:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572548.343245:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.343251:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343258:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb1649b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343271:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343277:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343283:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343289:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.343292:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343298:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343305:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0xa16187a3 (ffff9627d4b57f00) 00010000:00010000:1.0:1788572548.343307:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.343310:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343315:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1649be lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343350:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb1649b0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f06cd expref: 190 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343360:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:1.0:1788572548.343362:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572548.343366:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee400/0x85ace67feb1649b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f06cd expref: 190 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.343375:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc -2) 00000100:00100000:2.0:1788572548.343553:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:2350:x1875453360566272:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 8675us (9484us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572548.343564:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 365 00000100:00100000:2.0:1788572548.344615:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.344698:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.345212:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360566528 00000100:00100000:3.0:1788572548.345222:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360566528 00000100:00100000:3.0:1788572548.345279:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 322 00000100:00100000:3.0:1788572548.345289:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:2350:x1875453360566528:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.345313:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.345336:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb1649a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f06c6 expref: 188 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.345354:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa4:0x0].0x0 (ffff9626c1d8f300) 00010000:00010000:3.0:1788572548.345360:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.345409:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+187:2350:x1875453360566528:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 121us (709us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.345416:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 322 00000100:00100000:1.0:1788572548.346311:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360566400 00000100:00100000:1.0:1788572548.346319:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360566400 00002000:00100000:1.0:1788572548.346359:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x59:0x0] (0->4095) req@ffff9627ffc0d880 x1875453360566400/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.346376:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0d880 with x1875453360566400 ext(0->4095) 00002000:00010000:1.0:1788572548.346382:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ffc0d880 00000100:00100000:1.0:1788572548.346412:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 304 00000100:00100000:1.0:1788572548.346420:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2350:x1875453360566400:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.346631:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2350:x1875453360566400:12345-192.168.202.40@tcp:3:diff.0 Request processed in 211us (2019us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.346639:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 304 00002000:00100000:1.0:1788572548.346646:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x59:0x0] (0->4095) req@ffff9627ffc0d880 x1875453360566400/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.346654:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0d880 with x1875453360566400 ext(0->4095) 00002000:00010000:1.0:1788572548.346658:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ffc0d880 00000100:00100000:1.0:1788572548.356640:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.356842:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360566784 00000100:00100000:3.0:1788572548.356853:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360566784 00000100:00100000:3.0:1788572548.356918:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 988 00000100:00100000:3.0:1788572548.356928:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+187:35867:x1875453360566784:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572548.357359:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360566784:12345-192.168.202.40@tcp:35:diff.0 Request processed in 431us (727us total) trans 4294970844 rc 0/0 00000100:00100000:3.0:1788572548.357367:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 988 00000100:00100000:3.0:1788572548.362572:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.367769:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360566912 00000100:00100000:0.0:1788572548.367789:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360566912 00000100:00100000:2.0:1788572548.368087:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4733 00000100:00100000:2.0:1788572548.368099:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+188:35867:x1875453360566912:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.368126:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.368148:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1649c5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07b4 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.368220:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1649cc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.368236:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1649cc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.368248:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1649cc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.368284:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x97:0x0]: returning size 16384 00000004:00200000:2.0:1788572548.368287:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.368316:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1649cc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.368346:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb1649c5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07b4 expref: 190 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.368365:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04200/0x85ace67feb1649cc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07b4 expref: 189 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.368373:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.368376:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572548.368380:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04200, rc 0) 00000100:00100000:2.0:1788572548.368454:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+189:35867:x1875453360566912:12345-192.168.202.40@tcp:101:diff.0 Request processed in 357us (5893us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.368461:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4733 00000100:00100000:1.0:1788572548.378230:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.378391:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360567168 00000100:00100000:0.0:1788572548.378403:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360567168 00000100:00100000:0.0:1788572548.378476:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4734 00000100:00100000:0.0:1788572548.378486:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+189:35867:x1875453360567168:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.378508:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.378531:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb1649d3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07bb expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572548.378630:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x97:0x0]: returning size 16384 00010000:00010000:0.0:1788572548.379150:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb1649d3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07bb expref: 191 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.379166:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:0.0:1788572548.379169:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572548.379174:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb1649d3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07bb expref: 191 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.379187:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92200, rc 0) 00000100:00100000:0.0:1788572548.379257:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360567168:12345-192.168.202.40@tcp:101:diff.0 Request processed in 772us (1038us total) trans 4294970845 rc 301/301 00000100:00100000:0.0:1788572548.379266:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4734 00000100:00100000:1.0:1788572548.384782:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.384893:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360567296 00000100:00100000:2.0:1788572548.384913:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360567296 00000100:00100000:2.0:1788572548.384971:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 989 00000100:00100000:2.0:1788572548.384980:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360567296:12345-192.168.202.40@tcp:37:diff.0 00000100:00100000:2.0:1788572548.390169:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360567296:12345-192.168.202.40@tcp:37:diff.0 Request processed in 5177us (5385us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.390196:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 989 00000100:00100000:0.0:1788572548.394449:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.394543:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360567552 00000100:00100000:2.0:1788572548.394554:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360567552 00000100:00100000:2.0:1788572548.394669:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 990 00000100:00100000:2.0:1788572548.394681:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+190:35867:x1875453360567552:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572548.394968:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:35867:x1875453360567552:12345-192.168.202.40@tcp:35:diff.0 Request processed in 288us (541us total) trans 4294970846 rc 0/0 00000100:00100000:2.0:1788572548.394977:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 990 00000100:00100000:2.0:1788572548.398278:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.398364:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360567680 00000100:00100000:3.0:1788572548.398373:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360567680 00000100:00100000:3.0:1788572548.398455:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4735 00000100:00100000:3.0:1788572548.398466:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+191:35867:x1875453360567680:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.398490:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.398510:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1649e1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07c2 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398579:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398595:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398606:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398622:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398629:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398636:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398974:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1649f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398984:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1649f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.398992:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1649f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.399067:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa1:0x0]: returning size 3 00000004:00200000:3.0:1788572548.399072:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.399094:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1649f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399107:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1649f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399116:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399122:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399128:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399133:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.399137:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399146:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399158:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:3.0:1788572548.399161:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.399166:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399172:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb1649e8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399185:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399191:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399197:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399202:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb14d200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.399205:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399211:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399219:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x9beb5ca3 (ffff9626c1d8f900) 00010000:00010000:3.0:1788572548.399222:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.399225:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399231:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d200/0x85ace67feb1649ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399265:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb1649e1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07c2 expref: 193 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399281:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14c600/0x85ace67feb1649f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07c2 expref: 192 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.399288:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa1:0x0].0x0 (ffff9626c1d8fa00) 00010000:00010000:3.0:1788572548.399290:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.399293:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14c600, rc 0) 00000100:00100000:3.0:1788572548.399350:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+192:35867:x1875453360567680:12345-192.168.202.40@tcp:101:diff.0 Request processed in 885us (1083us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.399357:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4735 00000100:00100000:0.0:1788572548.403630:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.403711:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360567808 00000100:00100000:3.0:1788572548.403720:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360567808 00000100:00100000:3.0:1788572548.403778:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1350 00000100:00100000:3.0:1788572548.403787:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2349:x1875453360567808:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.403805:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.403826:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005838716c/0x85ace67feb1649fd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07c9 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.403852:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x35:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572548.403873:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x35:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.403877:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x35:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.403888:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005838716c/0x85ace67feb1649fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07c9 expref: 58 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.403915:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005838716c/0x85ace67feb1649fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07c9 expref: 58 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.403929:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc200, rc 0) 00000100:00100000:3.0:1788572548.403992:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2349:x1875453360567808:12345-192.168.202.40@tcp:101:diff.0 Request processed in 205us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.404000:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1350 00000100:00100000:0.0:1788572548.407897:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.408362:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360568064 00000100:00100000:2.0:1788572548.408374:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360568064 00000100:00100000:2.0:1788572548.408447:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4736 00000100:00100000:2.0:1788572548.408458:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+192:35867:x1875453360568064:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.408481:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.408506:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a04 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07d0 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.408622:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a0b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.408638:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.408651:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.408664:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa1:0x0]: returning size 3 00010000:00010000:2.0:1788572548.408808:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a04 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07d0 expref: 195 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.408825:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a0b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07d0 expref: 194 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.408834:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa1:0x0].0x0 (ffff9626c1d8fa00) 00010000:00010000:2.0:1788572548.408837:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.408841:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05800, rc 0) 00000100:00100000:2.0:1788572548.408914:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+194:35867:x1875453360568064:12345-192.168.202.40@tcp:101:diff.0 Request processed in 449us (1018us total) trans 4294970847 rc 0/0 00000100:00100000:2.0:1788572548.408921:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4736 00000100:00100000:0.0:1788572548.410322:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788572548.410345:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382393984:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788572548.410418:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572548.410541:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382393984 00000100:00100000:1.0:1788572548.410549:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382393984 00000100:00100000:1.0:1788572548.410604:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:1.0:1788572548.410613:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382393984:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788572548.410633:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3856662528 avail 3849322496 00000100:00100000:1.0:1788572548.410685:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382393984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.410692:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1788572548.411332:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382393984:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788572548.416086:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.416139:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.416174:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360568320 00000100:00100000:2.0:1788572548.416193:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360568320 00000100:00100000:2.0:1788572548.416257:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 323 00000100:00100000:2.0:1788572548.416268:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2effc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+194:2349:x1875453360568320:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.416285:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.416311:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07d0 expref: 194 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.416331:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa1:0x0].0x0 (ffff9626c1d8fa00) 00010000:00010000:2.0:1788572548.416337:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572548.416394:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2effc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+193:2349:x1875453360568320:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 128us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.416402:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 323 00000100:00100000:3.0:1788572548.416809:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360568192 00000100:00100000:3.0:1788572548.416816:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360568192 00002000:00100000:3.0:1788572548.416855:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x35:0x0] (0->4095) req@ffff9627ffc0e680 x1875453360568192/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.416866:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0e680 with x1875453360568192 ext(0->4095) 00002000:00010000:3.0:1788572548.416871:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ffc0e680 00000100:00100000:3.0:1788572548.416902:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 305 00000100:00100000:3.0:1788572548.416910:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2350:x1875453360568192:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572548.417193:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2350:x1875453360568192:12345-192.168.202.40@tcp:3:diff.0 Request processed in 283us (1118us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572548.417200:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 305 00002000:00100000:3.0:1788572548.417207:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x35:0x0] (0->4095) req@ffff9627ffc0e680 x1875453360568192/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.417217:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0e680 with x1875453360568192 ext(0->4095) 00002000:00010000:3.0:1788572548.417220:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ffc0e680 00000100:00100000:2.0:1788572548.421950:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.422098:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360568576 00000100:00100000:1.0:1788572548.422109:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360568576 00000100:00100000:1.0:1788572548.422167:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 991 00000100:00100000:1.0:1788572548.422177:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+193:35867:x1875453360568576:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.422638:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+194:35867:x1875453360568576:12345-192.168.202.40@tcp:35:diff.0 Request processed in 461us (697us total) trans 4294970848 rc 0/0 00000100:00100000:1.0:1788572548.422647:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 991 00000100:00100000:0.0:1788572548.426758:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.426867:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360568704 00000100:00100000:3.0:1788572548.426880:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360568704 00000100:00100000:3.0:1788572548.426954:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4737 00000100:00100000:3.0:1788572548.426965:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+194:35867:x1875453360568704:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.426998:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.427063:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164a19 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07d7 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427140:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427157:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427169:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427196:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427203:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427211:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427539:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164a2e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x98:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427548:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164a2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427559:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164a2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.427593:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x98:0x0]: returning size 3 00000004:00200000:3.0:1788572548.427597:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.427618:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164a2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427631:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164a2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427639:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427645:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427652:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427657:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.427662:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427669:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427680:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:3.0:1788572548.427683:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.427688:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427698:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cde00/0x85ace67feb164a20 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427709:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427715:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427720:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427725:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.427728:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427734:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427741:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x6e9b5ca3 (ffff9626c1d8f900) 00010000:00010000:3.0:1788572548.427743:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.427746:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427751:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a27 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427785:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164a19 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07d7 expref: 196 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427805:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cd800/0x85ace67feb164a2e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07d7 expref: 195 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.427811:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x98:0x0].0x0 (ffff9626c1d8f700) 00010000:00010000:3.0:1788572548.427813:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.427815:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cd800, rc 0) 00000100:00100000:3.0:1788572548.427887:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+195:35867:x1875453360568704:12345-192.168.202.40@tcp:101:diff.0 Request processed in 923us (1140us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.427895:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4737 00000100:00100000:2.0:1788572548.432588:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.432664:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360568832 00000100:00100000:1.0:1788572548.432674:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360568832 00000100:00100000:1.0:1788572548.432729:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1351 00000100:00100000:1.0:1788572548.432738:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+46:2349:x1875453360568832:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.432755:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.432779:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f61d707e/0x85ace67feb164a35 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x33:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07de expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572548.432811:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x33:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:1.0:1788572548.432831:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x33:0x0].0x0 since=896s v=(0 0) 00010000:00010000:1.0:1788572548.432836:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x33:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572548.432857:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f61d707e/0x85ace67feb164a35 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x33:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07de expref: 47 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.432876:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f61d707e/0x85ace67feb164a35 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x33:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07de expref: 47 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.432892:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee400, rc 0) 00000100:00100000:1.0:1788572548.433127:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2349:x1875453360568832:12345-192.168.202.40@tcp:101:diff.0 Request processed in 389us (549us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.433134:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1351 00000100:00100000:2.0:1788572548.437355:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.437603:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360569088 00000100:00100000:3.0:1788572548.437615:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360569088 00000100:00100000:3.0:1788572548.437676:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4738 00000100:00100000:3.0:1788572548.437685:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+195:35867:x1875453360569088:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.437706:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.437725:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164a3c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07e5 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.437825:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a43 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x98:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.437839:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.437864:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.437875:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x98:0x0]: returning size 3 00010000:00010000:3.0:1788572548.438054:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164a3c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07e5 expref: 198 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.438072:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a43 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07e5 expref: 197 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.438081:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x98:0x0].0x0 (ffff9626c1d8f700) 00010000:00010000:3.0:1788572548.438084:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.438087:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:3.0:1788572548.438143:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+197:35867:x1875453360569088:12345-192.168.202.40@tcp:101:diff.0 Request processed in 458us (797us total) trans 4294970849 rc 0/0 00000100:00100000:3.0:1788572548.438150:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4738 00000100:00100000:3.0:1788572548.442735:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.442825:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360569216 00000100:00100000:1.0:1788572548.442837:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360569216 00002000:00100000:1.0:1788572548.442893:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x33:0x0] (0->4095) req@ffff9626c4b33480 x1875453360569216/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.442905:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b33480 with x1875453360569216 ext(0->4095) 00002000:00010000:1.0:1788572548.442921:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b33480 00000100:00100000:1.0:1788572548.442957:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 306 00000100:00100000:1.0:1788572548.442968:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2350:x1875453360569216:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.443244:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2350:x1875453360569216:12345-192.168.202.40@tcp:3:diff.0 Request processed in 277us (519us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.443252:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 306 00002000:00100000:1.0:1788572548.443259:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x33:0x0] (0->4095) req@ffff9626c4b33480 x1875453360569216/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.443267:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b33480 with x1875453360569216 ext(0->4095) 00002000:00010000:1.0:1788572548.443270:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b33480 00000100:00100000:0.0:1788572548.445799:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.446195:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360569344 00000100:00100000:2.0:1788572548.446206:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360569344 00000100:00100000:2.0:1788572548.446281:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 324 00000100:00100000:2.0:1788572548.446292:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+197:2349:x1875453360569344:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.446309:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.446337:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a43 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x98:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07e5 expref: 197 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.446357:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x98:0x0].0x0 (ffff9626c1d8f700) 00010000:00010000:2.0:1788572548.446362:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572548.446428:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+196:2349:x1875453360569344:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 137us (639us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.446445:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 324 00000100:00100000:2.0:1788572548.451651:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.451796:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360569600 00000100:00100000:0.0:1788572548.451808:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360569600 00000100:00100000:0.0:1788572548.451883:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 992 00000100:00100000:0.0:1788572548.451893:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+196:35867:x1875453360569600:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572548.452531:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+197:35867:x1875453360569600:12345-192.168.202.40@tcp:35:diff.0 Request processed in 638us (888us total) trans 4294970850 rc 0/0 00000100:00100000:0.0:1788572548.452541:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 992 00000100:00100000:0.0:1788572548.455983:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.456264:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360569728 00000100:00100000:2.0:1788572548.456277:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360569728 00000100:00100000:2.0:1788572548.456350:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4739 00000100:00100000:2.0:1788572548.456366:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+197:35867:x1875453360569728:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.456401:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.456463:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a51 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07ec expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.456554:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.456572:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.456589:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.456606:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.456614:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.456621:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457070:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164a66 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457080:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164a66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457087:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164a66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.457122:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa0:0x0]: returning size 3 00000004:00200000:2.0:1788572548.457126:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.457147:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164a66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457159:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164a66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457167:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457178:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457184:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457189:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.457193:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457200:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457212:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.457215:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.457220:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457226:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164a58 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457237:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457242:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457247:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457252:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.457255:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457261:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457268:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xe44515a3 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572548.457270:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.457273:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457279:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164a5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457307:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164a51 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07ec expref: 199 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457321:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04800/0x85ace67feb164a66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07ec expref: 198 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.457328:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa0:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:2.0:1788572548.457330:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.457332:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04800, rc 0) 00000100:00100000:2.0:1788572548.457473:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+198:35867:x1875453360569728:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1111us (1504us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.457481:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4739 00000100:00100000:2.0:1788572548.462155:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.462322:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360569856 00000100:00100000:3.0:1788572548.462333:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360569856 00000100:00100000:3.0:1788572548.462421:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1352 00000100:00100000:3.0:1788572548.462442:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+47:2349:x1875453360569856:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.462462:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.462485:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb164a6d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07f3 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.462516:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x37:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572548.462540:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x37:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.462545:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x37:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.462557:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb164a6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07f3 expref: 48 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.462583:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000002eb965a/0x85ace67feb164a6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x37:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07f3 expref: 48 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.462597:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cde00, rc 0) 00000100:00100000:3.0:1788572548.462661:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2349:x1875453360569856:12345-192.168.202.40@tcp:101:diff.0 Request processed in 231us (812us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.462668:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1352 00000100:00100000:1.0:1788572548.466262:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.469047:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360570112 00000100:00100000:0.0:1788572548.469063:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360570112 00000100:00100000:0.0:1788572548.469475:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4740 00000100:00100000:0.0:1788572548.469488:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+198:35867:x1875453360570112:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.469510:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.469533:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164a74 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f07fa expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.469640:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164a7b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.469655:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164a7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.469666:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164a7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572548.469677:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xa0:0x0]: returning size 3 00010000:00010000:0.0:1788572548.469819:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164a74 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07fa expref: 201 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.469844:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164a7b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07fa expref: 200 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.469867:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa0:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:0.0:1788572548.469871:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572548.469875:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92a00, rc 0) 00000100:00100000:0.0:1788572548.470139:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+200:35867:x1875453360570112:12345-192.168.202.40@tcp:101:diff.0 Request processed in 653us (3886us total) trans 4294970851 rc 0/0 00000100:00100000:0.0:1788572548.470148:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4740 00000100:00100000:2.0:1788572548.476723:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.476778:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.476818:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360570368 00000100:00100000:2.0:1788572548.476830:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360570368 00000100:00100000:2.0:1788572548.476929:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 325 00000100:00100000:2.0:1788572548.476941:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+200:2350:x1875453360570368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.476962:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.476998:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164a7b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f07fa expref: 200 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.477050:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa0:0x0].0x0 (ffff9627c3dbee00) 00010000:00010000:2.0:1788572548.477056:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572548.477119:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+199:2350:x1875453360570368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 179us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.477127:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 325 00000100:00100000:1.0:1788572548.477562:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360570240 00000100:00100000:1.0:1788572548.477568:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360570240 00002000:00100000:1.0:1788572548.477607:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x37:0x0] (0->4095) req@ffff9627ffc0df80 x1875453360570240/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.477617:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0df80 with x1875453360570240 ext(0->4095) 00002000:00010000:1.0:1788572548.477622:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ffc0df80 00000100:00100000:1.0:1788572548.477652:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 307 00000100:00100000:1.0:1788572548.477660:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2349:x1875453360570240:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.478408:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2349:x1875453360570240:12345-192.168.202.40@tcp:3:diff.0 Request processed in 743us (1690us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.478418:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 307 00002000:00100000:1.0:1788572548.478426:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x37:0x0] (0->4095) req@ffff9627ffc0df80 x1875453360570240/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.478447:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ffc0df80 with x1875453360570240 ext(0->4095) 00002000:00010000:1.0:1788572548.478452:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ffc0df80 00000100:00100000:2.0:1788572548.483226:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.483462:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360570624 00000100:00100000:1.0:1788572548.483474:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360570624 00000100:00100000:1.0:1788572548.483538:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 993 00000100:00100000:1.0:1788572548.483549:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+199:35867:x1875453360570624:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.484308:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+200:35867:x1875453360570624:12345-192.168.202.40@tcp:35:diff.0 Request processed in 755us (1092us total) trans 4294970852 rc 0/0 00000100:00100000:1.0:1788572548.484320:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 993 00000100:00100000:2.0:1788572548.488253:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.488337:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360570752 00000100:00100000:3.0:1788572548.488348:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360570752 00000100:00100000:3.0:1788572548.488417:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4741 00000100:00100000:3.0:1788572548.488427:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+200:35867:x1875453360570752:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.488466:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.488490:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164a89 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0801 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.488566:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.488582:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.488594:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.488609:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.488620:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.488626:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489192:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164a9e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489203:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164a9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489210:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164a9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.489243:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9f:0x0]: returning size 3 00000004:00200000:3.0:1788572548.489248:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572548.489266:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164a9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489277:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164a9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489285:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489292:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489298:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489308:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.489313:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489321:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489331:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:3.0:1788572548.489334:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.489339:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489345:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164a90 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489356:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489362:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489368:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489373:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572548.489376:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489381:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489392:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x2c9ecea3 (ffff9626c1d8f800) 00010000:00010000:3.0:1788572548.489394:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572548.489397:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489403:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc400/0x85ace67feb164a97 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489442:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cc800/0x85ace67feb164a89 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0801 expref: 202 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489455:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14cc00/0x85ace67feb164a9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0801 expref: 201 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.489461:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9f:0x0].0x0 (ffff9626c1d8f100) 00010000:00010000:3.0:1788572548.489464:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.489466:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14cc00, rc 0) 00000100:00100000:3.0:1788572548.489607:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+201:35867:x1875453360570752:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1180us (1366us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.489615:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4741 00000100:00100000:2.0:1788572548.493442:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.493524:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360570880 00000100:00100000:3.0:1788572548.493536:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360570880 00000100:00100000:3.0:1788572548.493601:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1353 00000100:00100000:3.0:1788572548.493612:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+48:2349:x1875453360570880:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.493631:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.493654:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb164aa5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x34:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0808 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.493688:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x34:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572548.493708:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x34:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.493712:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x34:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.493723:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb164aa5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x34:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0808 expref: 49 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.493740:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef924bbe/0x85ace67feb164aa5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x34:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0808 expref: 49 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.493754:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d200, rc 0) 00000100:00100000:3.0:1788572548.493815:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2349:x1875453360570880:12345-192.168.202.40@tcp:101:diff.0 Request processed in 205us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.493822:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1353 00000100:00100000:0.0:1788572548.502718:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.502819:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360571136 00000100:00100000:2.0:1788572548.502830:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360571136 00000100:00100000:2.0:1788572548.502892:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4742 00000100:00100000:2.0:1788572548.502911:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+201:35867:x1875453360571136:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.502935:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.502962:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164aac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f080f expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.503133:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164ab3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.503151:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164ab3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.503173:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164ab3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.503187:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9f:0x0]: returning size 3 00010000:00010000:2.0:1788572548.503362:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164aac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f080f expref: 204 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.503388:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164ab3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f080f expref: 203 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.503398:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9f:0x0].0x0 (ffff9626c1d8f100) 00010000:00010000:2.0:1788572548.503402:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.503407:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05800, rc 0) 00000100:00100000:2.0:1788572548.503504:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+203:35867:x1875453360571136:12345-192.168.202.40@tcp:101:diff.0 Request processed in 601us (794us total) trans 4294970853 rc 0/0 00000100:00100000:2.0:1788572548.503513:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4742 00000100:00100000:2.0:1788572548.511677:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.512132:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360571264 00000100:00100000:3.0:1788572548.512143:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360571264 00000100:00100000:3.0:1788572548.512212:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 326 00000100:00100000:3.0:1788572548.512222:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+203:2348:x1875453360571264:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.512238:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.512267:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164ab3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f080f expref: 203 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.512288:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9f:0x0].0x0 (ffff9626c1d8f100) 00010000:00010000:3.0:1788572548.512294:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.512347:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+202:2348:x1875453360571264:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 126us (690us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.512355:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 326 00000100:00100000:0.0:1788572548.513058:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.513188:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360571392 00000100:00100000:1.0:1788572548.513195:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360571392 00002000:00100000:1.0:1788572548.513232:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x34:0x0] (0->4095) req@ffff9626c4920000 x1875453360571392/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.513243:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4920000 with x1875453360571392 ext(0->4095) 00002000:00010000:1.0:1788572548.513248:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4920000 00000100:00100000:1.0:1788572548.513275:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 308 00000100:00100000:1.0:1788572548.513283:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2350:x1875453360571392:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.513522:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2350:x1875453360571392:12345-192.168.202.40@tcp:3:diff.0 Request processed in 238us (465us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.513530:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 308 00002000:00100000:1.0:1788572548.513536:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x34:0x0] (0->4095) req@ffff9626c4920000 x1875453360571392/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.513546:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4920000 with x1875453360571392 ext(0->4095) 00002000:00010000:1.0:1788572548.513549:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4920000 00000100:00100000:0.0:1788572548.524106:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.524288:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360571648 00000100:00100000:1.0:1788572548.524300:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360571648 00000100:00100000:1.0:1788572548.524366:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 994 00000100:00100000:1.0:1788572548.524377:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+202:35867:x1875453360571648:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.524830:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+203:35867:x1875453360571648:12345-192.168.202.40@tcp:35:diff.0 Request processed in 453us (733us total) trans 4294970854 rc 0/0 00000100:00100000:1.0:1788572548.524839:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 994 00000100:00100000:0.0:1788572548.536113:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.536224:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360571776 00000100:00100000:1.0:1788572548.536235:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360571776 00000100:00100000:1.0:1788572548.536336:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4743 00000100:00100000:1.0:1788572548.536348:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+203:35867:x1875453360571776:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.536375:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.536398:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164ac1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0816 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536487:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536502:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536514:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536532:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536539:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536547:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536846:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb164ad6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536856:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb164ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.536863:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb164ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572548.536898:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9e:0x0]: returning size 3 00000004:00200000:1.0:1788572548.538378:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572548.538428:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb164ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538458:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb164ad6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538469:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538476:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538483:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538493:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23eee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.538498:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538506:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538520:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572548.538524:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.538528:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538534:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164ac8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538549:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538554:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538559:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538564:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c37d7000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572548.538567:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538572:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538582:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x74f887a3 (ffff9627d4b57100) 00010000:00010000:1.0:1788572548.538584:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572548.538587:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538592:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164acf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538630:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164ac1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0816 expref: 205 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538644:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d6400/0x85ace67feb164ad6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0816 expref: 204 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.538651:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9e:0x0].0x0 (ffff9627d4b57400) 00010000:00010000:1.0:1788572548.538653:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572548.538655:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c37d6400, rc 0) 00000100:00100000:1.0:1788572548.538737:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+204:35867:x1875453360571776:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2390us (2632us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.538746:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4743 00000100:00100000:2.0:1788572548.550444:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.551945:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360571904 00000100:00100000:0.0:1788572548.551963:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360571904 00000100:00100000:0.0:1788572548.552062:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1354 00000100:00100000:0.0:1788572548.552072:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+49:2349:x1875453360571904:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.552090:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.552112:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab805191/0x85ace67feb164add lrc: 2/0,0 mode: --/PR res: [0x240000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f081d expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572548.552145:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x35:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:0.0:1788572548.552171:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x35:0x0].0x0 since=896s v=(0 0) 00010000:00010000:0.0:1788572548.552176:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x35:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572548.552422:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab805191/0x85ace67feb164add lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f081d expref: 50 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.552453:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ab805191/0x85ace67feb164add lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x35:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f081d expref: 50 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.552466:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92400, rc 0) 00000100:00100000:0.0:1788572548.552536:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2349:x1875453360571904:12345-192.168.202.40@tcp:101:diff.0 Request processed in 466us (2115us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.552543:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1354 00000100:00100000:1.0:1788572548.557231:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.557320:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360572160 00000100:00100000:2.0:1788572548.557330:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360572160 00000100:00100000:2.0:1788572548.557395:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4744 00000100:00100000:2.0:1788572548.557405:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+204:35867:x1875453360572160:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.557426:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.557481:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164ae4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0824 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.557587:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164aeb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.557602:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164aeb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.557614:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164aeb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.557626:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9e:0x0]: returning size 3 00010000:00010000:2.0:1788572548.557757:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164ae4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0824 expref: 207 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.557777:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164aeb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0824 expref: 206 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.557787:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9e:0x0].0x0 (ffff9627d4b57400) 00010000:00010000:2.0:1788572548.557790:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.557794:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05800, rc 0) 00000100:00100000:2.0:1788572548.557863:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+206:35867:x1875453360572160:12345-192.168.202.40@tcp:101:diff.0 Request processed in 458us (639us total) trans 4294970855 rc 0/0 00000100:00100000:2.0:1788572548.557870:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4744 00000100:00100000:2.0:1788572548.563639:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.568305:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360572288 00000100:00100000:1.0:1788572548.568323:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360572288 00000100:00100000:1.0:1788572548.568400:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 327 00000100:00100000:1.0:1788572548.568412:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+206:2350:x1875453360572288:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572548.568429:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572548.568470:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164aeb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0824 expref: 206 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.568494:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9e:0x0].0x0 (ffff9627d4b57400) 00010000:00010000:1.0:1788572548.568500:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572548.568569:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+205:2350:x1875453360572288:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 159us (4941us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.568577:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 327 00000100:00100000:1.0:1788572548.569819:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.569966:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360572416 00000100:00100000:0.0:1788572548.569975:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360572416 00002000:00100000:0.0:1788572548.570060:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x35:0x0] (0->4095) req@ffff9626c1b58700 x1875453360572416/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.570072:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b58700 with x1875453360572416 ext(0->4095) 00002000:00010000:0.0:1788572548.570078:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b58700 00000100:00100000:0.0:1788572548.570103:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 309 00000100:00100000:0.0:1788572548.570113:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2350:x1875453360572416:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572548.570376:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2350:x1875453360572416:12345-192.168.202.40@tcp:3:diff.0 Request processed in 264us (564us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572548.570384:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 309 00002000:00100000:0.0:1788572548.570391:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x35:0x0] (0->4095) req@ffff9626c1b58700 x1875453360572416/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.570398:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b58700 with x1875453360572416 ext(0->4095) 00002000:00010000:0.0:1788572548.570401:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b58700 00000100:00100000:0.0:1788572548.583265:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.583674:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360572672 00000100:00100000:2.0:1788572548.583686:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360572672 00000100:00100000:2.0:1788572548.583753:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 995 00000100:00100000:2.0:1788572548.583764:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+205:35867:x1875453360572672:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572548.584467:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+206:35867:x1875453360572672:12345-192.168.202.40@tcp:35:diff.0 Request processed in 700us (1207us total) trans 4294970856 rc 0/0 00000100:00100000:2.0:1788572548.584477:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 995 00000100:00100000:3.0:1788572548.599253:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.601703:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360572800 00000100:00100000:2.0:1788572548.601721:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360572800 00000100:00100000:2.0:1788572548.601880:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4745 00000100:00100000:2.0:1788572548.601892:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+206:35867:x1875453360572800:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.601930:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.601954:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164af9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f082b expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602049:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602064:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602075:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602089:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602095:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602102:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602524:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164b0e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602535:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164b0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602547:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164b0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.602582:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9d:0x0]: returning size 3 00000004:00200000:2.0:1788572548.602587:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.602606:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164b0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602618:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164b0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602625:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602631:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602638:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602644:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.602648:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602655:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602666:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.602669:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.602673:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602683:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b00 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602695:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602701:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602706:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602711:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.602713:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602718:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602724:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xbd5240a3 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572548.602726:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.602729:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602734:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602766:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164af9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f082b expref: 208 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602784:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05800/0x85ace67feb164b0e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f082b expref: 207 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.602791:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9d:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:2.0:1788572548.602793:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.602796:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05800, rc 0) 00000100:00100000:2.0:1788572548.602885:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+207:35867:x1875453360572800:12345-192.168.202.40@tcp:101:diff.0 Request processed in 995us (3641us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.602894:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4745 00000100:00100000:1.0:1788572548.609569:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.609693:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360572928 00000100:00100000:3.0:1788572548.609703:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360572928 00000100:00100000:3.0:1788572548.609766:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1355 00000100:00100000:3.0:1788572548.609776:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+50:2350:x1875453360572928:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.609796:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.609820:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe4dd9a/0x85ace67feb164b15 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0832 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.609848:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x36:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572548.609868:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x36:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.609873:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x36:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.609883:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe4dd9a/0x85ace67feb164b15 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0832 expref: 51 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.609912:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cbe4dd9a/0x85ace67feb164b15 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0832 expref: 51 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.609927:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc400, rc 0) 00000100:00100000:3.0:1788572548.609981:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+51:2350:x1875453360572928:12345-192.168.202.40@tcp:101:diff.0 Request processed in 206us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.609988:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1355 00000100:00100000:1.0:1788572548.626988:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.627227:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360573184 00000100:00100000:0.0:1788572548.627238:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360573184 00000100:00100000:0.0:1788572548.627303:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4746 00000100:00100000:0.0:1788572548.627314:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+207:35867:x1875453360573184:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.627335:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.627358:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164b1c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0839 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.627483:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164b23 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.627500:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164b23 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.627513:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164b23 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572548.627525:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9d:0x0]: returning size 3 00010000:00010000:0.0:1788572548.627666:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164b1c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0839 expref: 210 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.627687:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164b23 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0839 expref: 209 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.627696:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9d:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:0.0:1788572548.627699:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572548.627704:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92000, rc 0) 00000100:00100000:0.0:1788572548.627861:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+209:35867:x1875453360573184:12345-192.168.202.40@tcp:101:diff.0 Request processed in 549us (883us total) trans 4294970857 rc 0/0 00000100:00100000:0.0:1788572548.627873:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4746 00000100:00100000:3.0:1788572548.649763:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.649813:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.649906:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360573440 00000100:00100000:2.0:1788572548.649925:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360573440 00000100:00100000:2.0:1788572548.649987:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 328 00000100:00100000:2.0:1788572548.649996:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+209:2349:x1875453360573440:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.650061:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.650087:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164b23 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0839 expref: 209 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.650107:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9d:0x0].0x0 (ffff9627c3dbe000) 00010000:00010000:2.0:1788572548.650112:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572548.650165:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+208:2349:x1875453360573440:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 170us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.650172:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 328 00000100:00100000:1.0:1788572548.650585:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360573312 00000100:00100000:1.0:1788572548.650592:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360573312 00002000:00100000:1.0:1788572548.650636:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x36:0x0] (0->4095) req@ffff9627c84ec700 x1875453360573312/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.650647:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ec700 with x1875453360573312 ext(0->4095) 00002000:00010000:1.0:1788572548.650653:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84ec700 00000100:00100000:1.0:1788572548.650687:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 310 00000100:00100000:1.0:1788572548.650695:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+51:2349:x1875453360573312:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.650986:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+51:2349:x1875453360573312:12345-192.168.202.40@tcp:3:diff.0 Request processed in 291us (1233us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.650994:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 310 00002000:00100000:1.0:1788572548.651001:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x36:0x0] (0->4095) req@ffff9627c84ec700 x1875453360573312/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.651034:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ec700 with x1875453360573312 ext(0->4095) 00002000:00010000:1.0:1788572548.651038:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627c84ec700 00000100:00100000:3.0:1788572548.668992:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.669160:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360573696 00000100:00100000:1.0:1788572548.669170:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360573696 00000100:00100000:1.0:1788572548.669244:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 996 00000100:00100000:1.0:1788572548.669255:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03a050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+208:35867:x1875453360573696:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.669728:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03a050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+209:35867:x1875453360573696:12345-192.168.202.40@tcp:35:diff.0 Request processed in 472us (745us total) trans 4294970858 rc 0/0 00000100:00100000:1.0:1788572548.669737:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 996 00000100:00100000:0.0:1788572548.678196:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.678415:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360573824 00000100:00100000:2.0:1788572548.678428:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360573824 00000100:00100000:2.0:1788572548.678504:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4747 00000100:00100000:2.0:1788572548.678515:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+209:35867:x1875453360573824:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.678541:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.678564:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164b31 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0840 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678634:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678649:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678661:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678675:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678682:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678687:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678989:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164b46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.678998:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164b46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679044:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164b46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.679083:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9c:0x0]: returning size 3 00000004:00200000:2.0:1788572548.679087:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.679107:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164b46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679119:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164b46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679127:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679132:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679139:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679145:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.679149:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679157:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679169:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.679172:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.679178:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679195:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b38 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679211:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679218:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679224:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679229:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.679231:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679237:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679244:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x5abf9a3 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572548.679247:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.679249:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679255:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b3f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679283:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164b31 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0840 expref: 211 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679296:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05400/0x85ace67feb164b46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0840 expref: 210 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.679309:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9c:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:2.0:1788572548.679311:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.679313:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05400, rc 0) 00000100:00100000:2.0:1788572548.679372:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+210:35867:x1875453360573824:12345-192.168.202.40@tcp:101:diff.0 Request processed in 858us (1197us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.679380:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4747 00000100:00100000:1.0:1788572548.687319:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.687366:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.687470:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574080 00000100:00100000:2.0:1788572548.687480:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574080 00000100:00100000:2.0:1788572548.687544:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 366 00000100:00100000:2.0:1788572548.687554:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+210:2348:x1875453360574080:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572548.687589:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.687607:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164b4d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f084e expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.687666:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.687680:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.687691:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.687706:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.687713:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.687720:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688064:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb164b62 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688074:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb164b62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688082:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb164b62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.688100:0:5842:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9b:0x0]: returning size 3 00000004:00200000:2.0:1788572548.688104:0:5842:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.688123:0:5842:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb164b62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688136:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb164b62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688144:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688150:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688156:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688161:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.688166:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688174:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688185:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.688188:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.688193:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688200:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164b54 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688210:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688216:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688221:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688226:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.688228:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688234:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688240:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x4e05b2a3 (ffff9627c3dbed00) 00010000:00010000:2.0:1788572548.688243:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.688246:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688255:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164b5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688284:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164b4d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f084e expref: 212 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688296:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bad600/0x85ace67feb164b62 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f084e expref: 211 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688303:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9b:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:2.0:1788572548.688305:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.688308:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bad600, rc 0) 00010000:00010000:2.0:1788572548.688314:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.688322:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164b69 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0855 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688346:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688354:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688361:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688372:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688379:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688385:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688563:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801fb9200/0x85ace67feb164b7e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688571:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801fb9200/0x85ace67feb164b7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688579:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801fb9200/0x85ace67feb164b7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.688593:0:5842:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9a:0x0]: returning size 3 00000004:00200000:2.0:1788572548.688597:0:5842:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.688606:0:5842:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801fb9200/0x85ace67feb164b7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688614:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801fb9200/0x85ace67feb164b7e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688620:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688625:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688630:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688634:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.688637:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688643:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688650:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.688652:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.688654:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688659:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164b70 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688669:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688675:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688680:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688684:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.688687:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688692:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688699:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x965f6ba3 (ffff9627c56ff200) 00010000:00010000:2.0:1788572548.688702:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.688704:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688715:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164b77 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688736:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164b69 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0855 expref: 213 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688748:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801fb9200/0x85ace67feb164b7e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0855 expref: 212 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688760:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9a:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:2.0:1788572548.688762:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.688765:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801fb9200, rc 0) 00010000:00010000:2.0:1788572548.688769:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.688778:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164b85 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f085c expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688799:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688807:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688815:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688825:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688832:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688839:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.688995:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f600/0x85ace67feb164b9a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x99:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689047:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f600/0x85ace67feb164b9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689056:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f600/0x85ace67feb164b9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.689070:0:5842:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x99:0x0]: returning size 3 00000004:00200000:2.0:1788572548.689073:0:5842:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.689082:0:5842:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f600/0x85ace67feb164b9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689090:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f600/0x85ace67feb164b9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689097:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689102:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689108:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689113:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478e400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.689115:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689121:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689129:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.689131:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.689133:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689138:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e400/0x85ace67feb164b8c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689147:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689153:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689160:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689165:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.689168:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689173:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689180:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xdeb924a3 (ffff9627e7a95700) 00010000:00010000:2.0:1788572548.689182:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.689185:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689194:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478e800/0x85ace67feb164b93 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689216:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164b85 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f085c expref: 214 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689225:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f600/0x85ace67feb164b9a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f085c expref: 213 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689231:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x99:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:2.0:1788572548.689233:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.689235:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478f600, rc 0) 00010000:00010000:2.0:1788572548.689239:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.689246:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164ba1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0863 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689268:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689275:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689281:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689291:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689298:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689304:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689380:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689387:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689394:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689399:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.689402:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689408:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689416:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.689419:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.689422:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689428:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164ba8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689450:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689456:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689463:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689468:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478f200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.689470:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689477:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689484:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xc8545ca3 (ffff9627d55e6200) 00010000:00010000:2.0:1788572548.689486:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.689488:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689494:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164baf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689517:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164ba1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0863 expref: 214 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689526:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:2.0:1788572548.689528:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572548.689531:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164ba1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0863 expref: 214 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.689540:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478fe00, rc -2) 00000100:00100000:2.0:1788572548.689604:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+213:2348:x1875453360574080:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2052us (2237us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572548.689616:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 366 00000100:00100000:3.0:1788572548.689836:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360573952 00000100:00100000:3.0:1788572548.689844:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360573952 00000100:00100000:3.0:1788572548.689912:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1356 00000100:00100000:3.0:1788572548.689921:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2349:x1875453360573952:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.689935:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.689951:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d19e11a3/0x85ace67feb164bb6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x34:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0847 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.689976:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x34:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572548.689993:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x34:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.689998:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x34:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.690034:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d19e11a3/0x85ace67feb164bb6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x34:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0847 expref: 59 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.690049:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d19e11a3/0x85ace67feb164bb6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x34:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0847 expref: 59 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.690062:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cc800, rc 0) 00000100:00100000:3.0:1788572548.690093:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+59:2349:x1875453360573952:12345-192.168.202.40@tcp:101:diff.0 Request processed in 174us (2784us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.690099:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1356 00000100:00100000:3.0:1788572548.695000:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.695739:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574336 00000100:00100000:0.0:1788572548.695749:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574336 00000100:00100000:0.0:1788572548.695814:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4748 00000100:00100000:0.0:1788572548.695824:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+213:35867:x1875453360574336:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.695845:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.695868:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164bbd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f087f expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.695969:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164bc4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.695983:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164bc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.695995:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164bc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572548.696037:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9c:0x0]: returning size 3 00010000:00010000:0.0:1788572548.696169:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92000/0x85ace67feb164bbd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f087f expref: 216 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.696267:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164bc4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f087f expref: 215 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.696277:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9c:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:0.0:1788572548.696279:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572548.696284:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92800, rc 0) 00000100:00100000:0.0:1788572548.696345:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+215:35867:x1875453360574336:12345-192.168.202.40@tcp:101:diff.0 Request processed in 523us (1356us total) trans 4294970859 rc 0/0 00000100:00100000:0.0:1788572548.696353:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4748 00000100:00100000:1.0:1788572548.701975:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.702136:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574464 00000100:00100000:2.0:1788572548.702149:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574464 00000100:00100000:2.0:1788572548.702213:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1357 00000100:00100000:2.0:1788572548.702224:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+59:2348:x1875453360574464:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:2.0:1788572548.702244:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.702266:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1a9ecec/0x85ace67feb164bd2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x32:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0886 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572548.702295:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x32:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:2.0:1788572548.702315:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x32:0x0].0x0 since=896s v=(0 0) 00010000:00010000:2.0:1788572548.702319:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x32:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.702329:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f1a9ecec/0x85ace67feb164bd2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x32:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0886 expref: 60 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.702347:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1a9ecec/0x85ace67feb164bd2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x32:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0886 expref: 60 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.702361:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478f000, rc 0) 00000100:00100000:2.0:1788572548.702416:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+60:2348:x1875453360574464:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 194us (452us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.702423:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1357 00000100:00100000:1.0:1788572548.703661:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.703747:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574592 00000100:00100000:3.0:1788572548.703754:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574592 00000100:00100000:3.0:1788572548.703802:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1358 00000100:00100000:3.0:1788572548.703812:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+60:2348:x1875453360574592:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572548.703828:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.703850:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a7ef4a3/0x85ace67feb164bd9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f088d expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572548.703872:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x36:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:3.0:1788572548.703890:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x36:0x0].0x0 since=896s v=(0 0) 00010000:00010000:3.0:1788572548.703895:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x36:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572548.703904:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004a7ef4a3/0x85ace67feb164bd9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f088d expref: 61 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.703921:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004a7ef4a3/0x85ace67feb164bd9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x36:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f088d expref: 61 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.703933:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c2477200, rc 0) 00000100:00100000:3.0:1788572548.703982:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+61:2348:x1875453360574592:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 173us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.703989:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1358 00000100:00100000:2.0:1788572548.708876:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.709884:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574720 00000100:00100000:0.0:1788572548.709897:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574720 00000100:00100000:0.0:1788572548.709961:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1359 00000100:00100000:0.0:1788572548.709972:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+61:2348:x1875453360574720:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572548.709989:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.710044:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000070b0cb4a/0x85ace67feb164be0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x33:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0894 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572548.710073:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x33:0x0] initial LVB size: 3, mtime: 0x6a9b7366, atime: 0x6a9b7366, ctime: 0x6a9b736e, blocks: 0xb 00010000:00010000:0.0:1788572548.710091:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x33:0x0].0x0 since=896s v=(0 0) 00010000:00010000:0.0:1788572548.710096:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x33:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572548.710106:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000070b0cb4a/0x85ace67feb164be0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x33:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0894 expref: 62 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.710123:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000070b0cb4a/0x85ace67feb164be0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x33:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0894 expref: 62 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.710137:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92000, rc 0) 00000100:00100000:0.0:1788572548.710213:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2348:x1875453360574720:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 242us (1347us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.710220:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1359 00000100:00100000:2.0:1788572548.715377:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.715426:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.715531:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574976 00000100:00100000:0.0:1788572548.715541:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574976 00000100:00100000:0.0:1788572548.715611:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 329 00000100:00100000:0.0:1788572548.715621:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+215:2350:x1875453360574976:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572548.715637:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572548.715662:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164bc4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f087f expref: 215 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.715682:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9c:0x0].0x0 (ffff9627c3dbe400) 00010000:00010000:0.0:1788572548.715687:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572548.715751:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+214:2350:x1875453360574976:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 130us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572548.715758:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 329 00000100:00100000:1.0:1788572548.716108:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360574848 00000100:00100000:1.0:1788572548.716114:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360574848 00002000:00100000:1.0:1788572548.716150:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x34:0x0] (0->4095) req@ffff9626c5432680 x1875453360574848/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.716161:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5432680 with x1875453360574848 ext(0->4095) 00002000:00010000:1.0:1788572548.716166:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5432680 00000100:00100000:1.0:1788572548.716195:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 311 00000100:00100000:1.0:1788572548.716204:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360574848:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.716511:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360574848:12345-192.168.202.40@tcp:3:diff.0 Request processed in 307us (1142us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.716520:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 311 00002000:00100000:1.0:1788572548.716527:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x34:0x0] (0->4095) req@ffff9626c5432680 x1875453360574848/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.716536:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5432680 with x1875453360574848 ext(0->4095) 00002000:00010000:1.0:1788572548.716539:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5432680 00000100:00100000:2.0:1788572548.721150:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.721334:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360575232 00000100:00100000:1.0:1788572548.721346:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360575232 00000100:00100000:1.0:1788572548.721407:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 997 00000100:00100000:1.0:1788572548.721418:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+214:35867:x1875453360575232:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.721868:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+215:35867:x1875453360575232:12345-192.168.202.40@tcp:35:diff.0 Request processed in 451us (727us total) trans 4294970860 rc 0/0 00000100:00100000:1.0:1788572548.721876:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 997 00000100:00100000:2.0:1788572548.727315:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.727422:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360575616 00000100:00100000:0.0:1788572548.727442:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360575616 00000100:00100000:0.0:1788572548.727508:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 367 00000100:00100000:0.0:1788572548.727519:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+215:2347:x1875453360575616:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572548.727583:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.727606:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164be7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f08a2 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727671:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727687:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727701:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727720:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727727:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727733:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727934:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727943:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727949:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727955:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.727959:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727967:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727978:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572548.727981:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.727987:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.727992:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164bee lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728042:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728049:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728055:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728060:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.728063:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728069:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728077:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x2f385ca3 (ffff9627fb1e7800) 00010000:00010000:0.0:1788572548.728079:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.728082:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728088:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164bf5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728122:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164be7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08a2 expref: 216 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.728132:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572548.728134:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572548.728139:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164be7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08a2 expref: 216 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.731520:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92800, rc -2) 00000100:00100000:0.0:1788572548.731638:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+215:2347:x1875453360575616:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 4116us (4327us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572548.731652:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 367 00000100:00100000:2.0:1788572548.736414:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.736511:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360575488 00000100:00100000:1.0:1788572548.736522:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360575488 00000100:00100000:1.0:1788572548.736597:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4749 00000100:00100000:1.0:1788572548.736609:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+215:35867:x1875453360575488:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.736638:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.736664:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164bfc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f089b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.736775:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c03 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.736791:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.736807:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572548.736823:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9b:0x0]: returning size 3 00010000:00010000:1.0:1788572548.736981:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164bfc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f089b expref: 218 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.737035:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c03 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f089b expref: 217 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.737045:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9b:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:1.0:1788572548.737048:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572548.737052:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c37d7c00, rc 0) 00000100:00100000:1.0:1788572548.737235:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+217:35867:x1875453360575488:12345-192.168.202.40@tcp:101:diff.0 Request processed in 626us (828us total) trans 4294970861 rc 0/0 00000100:00100000:1.0:1788572548.737244:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4749 00000100:00100000:2.0:1788572548.753257:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.753404:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360575744 00000100:00100000:3.0:1788572548.753414:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360575744 00000100:00100000:3.0:1788572548.753494:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 330 00000100:00100000:3.0:1788572548.753504:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efe850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+217:2348:x1875453360575744:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.753521:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.753549:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f089b expref: 217 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.753570:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9b:0x0].0x0 (ffff9627c3dbe100) 00010000:00010000:3.0:1788572548.753576:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.753638:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efe850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+216:2348:x1875453360575744:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 136us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.753645:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 330 00000100:00100000:2.0:1788572548.763082:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.763244:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360575872 00000100:00100000:3.0:1788572548.763255:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360575872 00002000:00100000:3.0:1788572548.763312:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x32:0x0] (0->4095) req@ffff9626c5430380 x1875453360575872/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.763324:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5430380 with x1875453360575872 ext(0->4095) 00002000:00010000:3.0:1788572548.763330:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5430380 00000100:00100000:3.0:1788572548.763367:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 312 00000100:00100000:3.0:1788572548.763377:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360575872:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:3.0:1788572548.763638:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360575872:12345-192.168.202.40@tcp:3:diff.0 Request processed in 261us (565us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788572548.763646:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 312 00002000:00100000:3.0:1788572548.763653:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x32:0x0] (0->4095) req@ffff9626c5430380 x1875453360575872/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572548.763661:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5430380 with x1875453360575872 ext(0->4095) 00002000:00010000:3.0:1788572548.763664:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5430380 00000100:00100000:1.0:1788572548.775644:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.776053:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360576128 00000100:00100000:2.0:1788572548.776065:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360576128 00000100:00100000:2.0:1788572548.776127:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 998 00000100:00100000:2.0:1788572548.776138:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+216:35867:x1875453360576128:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572548.776614:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+217:35867:x1875453360576128:12345-192.168.202.40@tcp:35:diff.0 Request processed in 476us (979us total) trans 4294970862 rc 0/0 00000100:00100000:2.0:1788572548.776623:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 998 00000100:00100000:1.0:1788572548.785243:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.785301:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.785516:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360576512 00000100:00100000:0.0:1788572548.785530:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360576512 00000100:00100000:0.0:1788572548.785604:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 368 00000100:00100000:0.0:1788572548.785614:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+217:2348:x1875453360576512:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:3.0:1788572548.785790:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360576384 00000100:00100000:3.0:1788572548.785794:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360576384 00000100:00100000:3.0:1788572548.785835:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4750 00000100:00100000:3.0:1788572548.785843:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+219:35867:x1875453360576384:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572548.785855:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572548.785880:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164c11 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f08e1 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.786044:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164c18 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.786060:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164c18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.786073:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164c18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572548.786084:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9a:0x0]: returning size 3 00010000:00010000:3.0:1788572548.786243:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164c11 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08e1 expref: 222 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.786264:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164c18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08e1 expref: 221 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.786273:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9a:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572548.786276:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572548.786280:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:3.0:1788572548.786320:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:35867:x1875453360576384:12345-192.168.202.40@tcp:101:diff.0 Request processed in 478us (1087us total) trans 4294970863 rc 0/0 00000100:00100000:3.0:1788572548.786327:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4750 00010000:00010000:0.0:1788572548.793397:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.793426:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c26 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f08e8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793507:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793522:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793534:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793548:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793554:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793560:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793767:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793775:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793782:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793788:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.793792:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793800:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793813:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572548.793816:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.793822:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793828:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c2d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793841:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793847:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793853:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793858:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.793861:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793867:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793875:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x1dd915a3 (ffff9627fb1e7800) 00010000:00010000:0.0:1788572548.793878:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.793880:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.793886:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c34 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.795046:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c26 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08e8 expref: 221 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.795063:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572548.795066:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572548.795071:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c26 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08e8 expref: 221 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.795084:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92800, rc -2) 00000100:00100000:0.0:1788572548.795252:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+219:2348:x1875453360576512:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 9569us (9880us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572548.795264:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 368 00000100:00100000:2.0:1788572548.796283:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.796992:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360576640 00000100:00100000:3.0:1788572548.797028:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360576640 00000100:00100000:3.0:1788572548.797090:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 331 00000100:00100000:3.0:1788572548.797177:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+219:2348:x1875453360576640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572548.797193:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572548.797216:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164c18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f08e1 expref: 219 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572548.797237:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9a:0x0].0x0 (ffff9627e7a95200) 00010000:00010000:3.0:1788572548.797243:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.797343:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+218:2348:x1875453360576640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 169us (1064us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.797351:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 331 00000100:00100000:3.0:1788572548.811246:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.813307:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360576768 00000100:00100000:0.0:1788572548.813325:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360576768 00002000:00100000:0.0:1788572548.813386:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x36:0x0] (0->4095) req@ffff9626c12edc00 x1875453360576768/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.813398:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12edc00 with x1875453360576768 ext(0->4095) 00002000:00010000:0.0:1788572548.813405:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c12edc00 00000100:00100000:0.0:1788572548.813686:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 313 00000100:00100000:0.0:1788572548.813699:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360576768:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572548.813996:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360576768:12345-192.168.202.40@tcp:3:diff.0 Request processed in 299us (2757us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572548.814022:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 313 00002000:00100000:0.0:1788572548.814030:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x36:0x0] (0->4095) req@ffff9626c12edc00 x1875453360576768/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.814039:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12edc00 with x1875453360576768 ext(0->4095) 00002000:00010000:0.0:1788572548.814042:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c12edc00 00000100:00100000:2.0:1788572548.824454:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.825060:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360577024 00000100:00100000:3.0:1788572548.825072:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360577024 00000100:00100000:3.0:1788572548.825143:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 999 00000100:00100000:3.0:1788572548.825153:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+218:35867:x1875453360577024:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572548.825614:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+219:35867:x1875453360577024:12345-192.168.202.40@tcp:35:diff.0 Request processed in 461us (1181us total) trans 4294970864 rc 0/0 00000100:00100000:3.0:1788572548.825623:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 999 00000100:00100000:2.0:1788572548.835338:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.835581:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360577280 00000100:00100000:0.0:1788572548.835593:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360577280 00000100:00100000:0.0:1788572548.835673:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4751 00000100:00100000:0.0:1788572548.835685:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+219:35867:x1875453360577280:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572548.835713:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572548.835742:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c3b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f097b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.835888:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c42 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x99:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.835907:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.835922:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572548.835935:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x99:0x0]: returning size 3 00010000:00010000:0.0:1788572548.836124:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c3b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f097b expref: 222 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.836145:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f097b expref: 221 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.836154:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x99:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:0.0:1788572548.836157:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572548.836161:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93600, rc 0) 00000100:00100000:0.0:1788572548.836301:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:35867:x1875453360577280:12345-192.168.202.40@tcp:101:diff.0 Request processed in 617us (972us total) trans 4294970865 rc 0/0 00000100:00100000:0.0:1788572548.836310:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4751 00000100:00100000:2.0:1788572548.838536:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.838611:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360577408 00000100:00100000:0.0:1788572548.838621:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360577408 00000100:00100000:0.0:1788572548.838681:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 369 00000100:00100000:0.0:1788572548.838693:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c97ce450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:2347:x1875453360577408:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:3.0:1788572548.851618:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572548.851660:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572548.851697:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360577536 00000100:00100000:0.0:1788572548.851708:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360577536 00002000:00100000:0.0:1788572548.851764:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x33:0x0] (0->4095) req@ffff9626c12ed180 x1875453360577536/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.851775:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12ed180 with x1875453360577536 ext(0->4095) 00002000:00010000:0.0:1788572548.851781:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c12ed180 00000100:00100000:0.0:1788572548.851817:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 314 00000100:00100000:0.0:1788572548.851828:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360577536:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572548.852103:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360577536:12345-192.168.202.40@tcp:3:diff.0 Request processed in 276us (495us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572548.852111:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 314 00002000:00100000:0.0:1788572548.852118:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x33:0x0] (0->4095) req@ffff9626c12ed180 x1875453360577536/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572548.852126:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12ed180 with x1875453360577536 ext(0->4095) 00002000:00010000:0.0:1788572548.852129:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c12ed180 00000100:00100000:1.0:1788572548.857532:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360577664 00000100:00100000:1.0:1788572548.857549:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360577664 00000100:00100000:1.0:1788572548.865611:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 332 00000100:00100000:1.0:1788572548.865640:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+223:2349:x1875453360577664:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572548.865661:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572548.865690:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.865693:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164c42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x99:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f097b expref: 223 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.865707:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f095f expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.865711:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x99:0x0].0x0 (ffff9627d55e6d00) 00010000:00010000:1.0:1788572548.865716:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572548.865769:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.865782:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.865794:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.865810:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.865816:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.865823:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870564:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870586:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870595:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870601:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.870606:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870615:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870637:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572548.870640:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.870647:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870653:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164c57 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870667:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870674:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870680:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870686:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572548.870689:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870695:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870703:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xa16187a3 (ffff9627fb1e7800) 00010000:00010000:0.0:1788572548.870705:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572548.870708:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870714:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164c5e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870765:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c50 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f095f expref: 223 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870777:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572548.870779:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572548.870783:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164c50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f095f expref: 223 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572548.870792:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92800, rc -2) 00000100:00100000:1.0:1788572548.873889:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:2349:x1875453360577664:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8248us (22214us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.873904:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 332 00000100:00100000:3.0:1788572548.876327:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.876569:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360577920 00000100:00100000:1.0:1788572548.876580:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360577920 00000100:00100000:1.0:1788572548.876639:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1000 00000100:00100000:1.0:1788572548.876650:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:35867:x1875453360577920:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.877085:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+222:35867:x1875453360577920:12345-192.168.202.40@tcp:35:diff.0 Request processed in 434us (766us total) trans 4294970866 rc 0/0 00000100:00100000:1.0:1788572548.877093:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1000 00000100:00100000:0.0:1788572548.885194:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c97ce450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:2347:x1875453360577408:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 46480us (46643us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572548.885212:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 369 00000100:00100000:3.0:1788572548.885598:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.885772:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360578048 00000100:00100000:1.0:1788572548.885782:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360578048 00000100:00100000:1.0:1788572548.885838:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4752 00000100:00100000:1.0:1788572548.885848:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+221:35867:x1875453360578048:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.885869:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.886272:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164c65 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a4d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.886349:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c6c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.886365:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.886377:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572548.886475:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8c:0x0]: returning size 16384 00000004:00200000:1.0:1788572548.886480:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572548.886501:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.886535:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164c65 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a4d expref: 223 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.886553:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7c00/0x85ace67feb164c6c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a4d expref: 222 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.886561:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:1.0:1788572548.886564:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572548.886567:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c37d7c00, rc 0) 00000100:00100000:1.0:1788572548.886620:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+222:35867:x1875453360578048:12345-192.168.202.40@tcp:101:diff.0 Request processed in 773us (1023us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572548.886628:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4752 00000100:00100000:3.0:1788572548.912257:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.912541:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360578304 00000100:00100000:2.0:1788572548.912553:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360578304 00000100:00100000:2.0:1788572548.912623:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4753 00000100:00100000:2.0:1788572548.912635:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+222:35867:x1875453360578304:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.912661:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.912683:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164c73 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a54 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.912788:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8c:0x0]: returning size 16384 00010000:00010000:2.0:1788572548.912935:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164c73 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a54 expref: 224 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.912957:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:2.0:1788572548.912961:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572548.912967:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164c73 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a54 expref: 224 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.912983:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478f200, rc 0) 00000100:00100000:2.0:1788572548.913089:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+223:35867:x1875453360578304:12345-192.168.202.40@tcp:101:diff.0 Request processed in 456us (842us total) trans 4294970867 rc 301/301 00000100:00100000:2.0:1788572548.913256:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4753 00000100:00100000:3.0:1788572548.916832:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.916929:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360578432 00000100:00100000:2.0:1788572548.916939:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360578432 00000100:00100000:2.0:1788572548.916996:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1001 00000100:00100000:2.0:1788572548.917034:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+223:35867:x1875453360578432:12345-192.168.202.40@tcp:37:diff.0 00000100:00100000:2.0:1788572548.923837:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+223:35867:x1875453360578432:12345-192.168.202.40@tcp:37:diff.0 Request processed in 6820us (7002us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.923851:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1001 00000100:00100000:3.0:1788572548.932369:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.932485:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360578688 00000100:00100000:1.0:1788572548.932496:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360578688 00000100:00100000:1.0:1788572548.932556:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1002 00000100:00100000:1.0:1788572548.932567:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03b050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+223:35867:x1875453360578688:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572548.932820:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03b050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+224:35867:x1875453360578688:12345-192.168.202.40@tcp:35:diff.0 Request processed in 254us (462us total) trans 4294970868 rc 0/0 00000100:00100000:1.0:1788572548.932828:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1002 00000100:00100000:0.0:1788572548.942915:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.943067:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360578816 00000100:00100000:2.0:1788572548.943078:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360578816 00000100:00100000:2.0:1788572548.943240:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4754 00000100:00100000:2.0:1788572548.943254:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+224:35867:x1875453360578816:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.943281:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.943304:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164c81 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a5b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943379:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943398:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943412:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943426:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943444:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943451:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943744:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164c96 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x96:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943754:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943762:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572548.943799:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x96:0x0]: returning size 3 00000004:00200000:2.0:1788572548.943803:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572548.943823:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943837:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943850:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943856:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943862:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943868:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.943872:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943881:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.943893:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572548.943896:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.947696:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947709:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164c88 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947735:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947742:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947749:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947755:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c478fe00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572548.947759:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947766:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947786:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x9beb5ca3 (ffff9627d55e6200) 00010000:00010000:2.0:1788572548.947789:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572548.947793:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947799:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164c8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947865:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164c81 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a5b expref: 226 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947882:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ee00/0x85ace67feb164c96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a5b expref: 225 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.947889:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x96:0x0].0x0 (ffff9627d55e6900) 00010000:00010000:2.0:1788572548.947892:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572548.947894:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478ee00, rc 0) 00000100:00100000:2.0:1788572548.947976:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+225:35867:x1875453360578816:12345-192.168.202.40@tcp:101:diff.0 Request processed in 4723us (5063us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.947987:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4754 00000100:00100000:3.0:1788572548.952167:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.958606:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360578944 00000100:00100000:2.0:1788572548.958622:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360578944 00000100:00100000:2.0:1788572548.958703:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1360 00000100:00100000:2.0:1788572548.958714:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2349:x1875453360578944:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572548.958734:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572548.958758:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000977d2372/0x85ace67feb164c9d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x2c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a62 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572548.958788:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x2c:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:2.0:1788572548.958807:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2c:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572548.958812:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572548.958822:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000977d2372/0x85ace67feb164c9d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a62 expref: 63 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.958844:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000977d2372/0x85ace67feb164c9d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a62 expref: 63 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.958859:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478e800, rc 0) 00000100:00100000:2.0:1788572548.958936:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+63:2349:x1875453360578944:12345-192.168.202.40@tcp:101:diff.0 Request processed in 223us (6787us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572548.958943:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1360 00000100:00100000:3.0:1788572548.973844:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.974533:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360579200 00000100:00100000:1.0:1788572548.974545:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360579200 00000100:00100000:1.0:1788572548.974595:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4755 00000100:00100000:1.0:1788572548.974605:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+225:35867:x1875453360579200:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572548.974624:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572548.974647:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164ca4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a69 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.974776:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cab lrc: 3/1,0 mode: --/PR res: [0x200000401:0x96:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.974792:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.974804:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572548.974817:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x96:0x0]: returning size 3 00010000:00010000:1.0:1788572548.974980:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164ca4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a69 expref: 228 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.974998:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a69 expref: 227 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572548.975058:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x96:0x0].0x0 (ffff9627d55e6900) 00010000:00010000:1.0:1788572548.975062:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572548.975065:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3327800, rc 0) 00000100:00100000:1.0:1788572548.975129:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+227:35867:x1875453360579200:12345-192.168.202.40@tcp:101:diff.0 Request processed in 525us (1296us total) trans 4294970869 rc 0/0 00000100:00100000:1.0:1788572548.975137:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4755 00000100:00100000:3.0:1788572548.989690:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572548.993831:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360579328 00000100:00100000:2.0:1788572548.993852:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360579328 00000100:00100000:2.0:1788572548.993971:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 333 00000100:00100000:2.0:1788572548.993983:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+227:2350:x1875453360579328:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572548.994054:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572548.994087:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cab lrc: 0/0,0 mode: --/PR res: [0x200000401:0x96:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a69 expref: 227 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572548.994107:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x96:0x0].0x0 (ffff9627d55e6900) 00010000:00010000:2.0:1788572548.994113:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572548.994256:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+226:2350:x1875453360579328:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 273us (4577us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572548.994266:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 333 00000100:00100000:2.0:1788572548.997116:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572548.997398:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360579456 00000100:00100000:1.0:1788572548.997410:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360579456 00002000:00100000:1.0:1788572548.997472:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x2c:0x0] (0->4095) req@ffff9626c5431880 x1875453360579456/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/0 e 0 to 0 dl 1788572559 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.997483:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431880 with x1875453360579456 ext(0->4095) 00002000:00010000:1.0:1788572548.997490:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5431880 00000100:00100000:1.0:1788572548.997528:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 315 00000100:00100000:1.0:1788572548.997537:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+63:2350:x1875453360579456:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572548.997788:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+63:2350:x1875453360579456:12345-192.168.202.40@tcp:3:diff.0 Request processed in 249us (680us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572548.997797:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 315 00002000:00100000:1.0:1788572548.997804:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x2c:0x0] (0->4095) req@ffff9626c5431880 x1875453360579456/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:209/0 lens 488/4536 e 0 to 0 dl 1788572559 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572548.997812:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431880 with x1875453360579456 ext(0->4095) 00002000:00010000:1.0:1788572548.997815:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5431880 00000100:00100000:2.0:1788572549.009529:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.009660:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360579712 00000100:00100000:0.0:1788572549.009673:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360579712 00000100:00100000:0.0:1788572549.009759:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1003 00000100:00100000:0.0:1788572549.009771:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+226:35867:x1875453360579712:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572549.010339:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+227:35867:x1875453360579712:12345-192.168.202.40@tcp:35:diff.0 Request processed in 569us (824us total) trans 4294970870 rc 0/0 00000100:00100000:0.0:1788572549.010348:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1003 00000100:00080000:3.0:1788572549.010462:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788572549.013188:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.013292:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360579840 00000100:00100000:0.0:1788572549.013302:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360579840 00000100:00100000:0.0:1788572549.013361:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4756 00000100:00100000:0.0:1788572549.013373:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+227:35867:x1875453360579840:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.013400:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.013421:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164cb9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a70 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013511:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013526:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013536:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013551:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013558:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013569:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013892:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb164cce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013901:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb164cce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013909:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb164cce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.013941:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8d:0x0]: returning size 3 00000004:00200000:0.0:1788572549.013946:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.013965:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb164cce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013978:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb164cce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013986:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013992:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.013999:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014059:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.014063:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014071:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014083:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.014086:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.014090:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014097:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164cc0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014110:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014116:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014122:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014127:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.014130:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014137:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014144:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x6e9b5ca3 (ffff9627fb1e7800) 00010000:00010000:0.0:1788572549.014146:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.014149:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014155:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164cc7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014188:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164cb9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a70 expref: 229 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014202:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92e00/0x85ace67feb164cce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a70 expref: 228 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.014208:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8d:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572549.014210:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.014213:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92e00, rc 0) 00000100:00100000:0.0:1788572549.014407:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+228:35867:x1875453360579840:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1035us (1229us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.014415:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4756 00000100:00100000:2.0:1788572549.023089:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.023259:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360579968 00000100:00100000:3.0:1788572549.023270:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360579968 00000100:00100000:3.0:1788572549.023344:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1361 00000100:00100000:3.0:1788572549.023355:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+51:2348:x1875453360579968:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.023375:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.023396:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006b07f7d0/0x85ace67feb164cd5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a77 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572549.023423:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x2c:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:3.0:1788572549.023456:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2c:0x0].0x0 since=897s v=(0 0) 00010000:00010000:3.0:1788572549.023461:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572549.023471:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b07f7d0/0x85ace67feb164cd5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a77 expref: 52 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.023491:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006b07f7d0/0x85ace67feb164cd5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a77 expref: 52 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.023506:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c200, rc 0) 00000100:00100000:3.0:1788572549.023567:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2348:x1875453360579968:12345-192.168.202.40@tcp:101:diff.0 Request processed in 213us (486us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.023574:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1361 00000100:00080000:0.0:1788572549.023664:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572549.027358:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.027468:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360580224 00000100:00100000:0.0:1788572549.027480:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360580224 00000100:00100000:0.0:1788572549.027547:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4757 00000100:00100000:0.0:1788572549.027556:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+228:35867:x1875453360580224:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.027578:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.027600:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164cdc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a7e expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.027708:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164ce3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.027727:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.027739:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.027751:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8d:0x0]: returning size 3 00010000:00010000:0.0:1788572549.027894:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92a00/0x85ace67feb164cdc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a7e expref: 231 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.027923:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164ce3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a7e expref: 230 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.027932:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8d:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:0.0:1788572549.027935:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.027939:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92c00, rc 0) 00000100:00100000:0.0:1788572549.028000:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+230:35867:x1875453360580224:12345-192.168.202.40@tcp:101:diff.0 Request processed in 446us (653us total) trans 4294970871 rc 0/0 00000100:00100000:0.0:1788572549.028152:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4757 00000100:00100000:2.0:1788572549.038285:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.038336:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.038419:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360580352 00000100:00100000:0.0:1788572549.038438:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360580352 00002000:00100000:0.0:1788572549.038488:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x2c:0x0] (0->4095) req@ffff9626c5431180 x1875453360580352/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572549.038499:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431180 with x1875453360580352 ext(0->4095) 00002000:00010000:0.0:1788572549.038505:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c5431180 00000100:00100000:0.0:1788572549.038540:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 316 00000100:00100000:0.0:1788572549.038549:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2349:x1875453360580352:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572549.038765:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2349:x1875453360580352:12345-192.168.202.40@tcp:3:diff.0 Request processed in 217us (491us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572549.038774:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 316 00002000:00100000:0.0:1788572549.038780:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x2c:0x0] (0->4095) req@ffff9626c5431180 x1875453360580352/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572549.038789:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431180 with x1875453360580352 ext(0->4095) 00002000:00010000:0.0:1788572549.038793:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c5431180 00000100:00100000:3.0:1788572549.039364:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360580480 00000100:00100000:3.0:1788572549.039372:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360580480 00000100:00100000:3.0:1788572549.039422:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 334 00000100:00100000:3.0:1788572549.039448:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+230:2349:x1875453360580480:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572549.039461:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572549.039484:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164ce3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a7e expref: 230 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.039503:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8d:0x0].0x0 (ffff9627fb1e7700) 00010000:00010000:3.0:1788572549.039508:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572549.039545:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+229:2349:x1875453360580480:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 116us (1209us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.039552:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 334 00000100:00100000:2.0:1788572549.052145:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.052464:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360580736 00000100:00100000:1.0:1788572549.052475:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360580736 00000100:00100000:1.0:1788572549.052534:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1004 00000100:00100000:1.0:1788572549.052543:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+229:35867:x1875453360580736:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.053167:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+230:35867:x1875453360580736:12345-192.168.202.40@tcp:35:diff.0 Request processed in 620us (1029us total) trans 4294970872 rc 0/0 00000100:00100000:1.0:1788572549.053178:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1004 00000100:00100000:2.0:1788572549.058760:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.058858:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360580864 00000100:00100000:1.0:1788572549.058869:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360580864 00000100:00100000:1.0:1788572549.058939:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4758 00000100:00100000:1.0:1788572549.058950:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+230:35867:x1875453360580864:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.058976:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.059000:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cf1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a85 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059122:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059137:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059149:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059165:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059173:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059192:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059525:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164d06 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x95:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059534:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164d06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059540:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164d06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572549.059577:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x95:0x0]: returning size 3 00000004:00200000:1.0:1788572549.059581:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572549.059600:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164d06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059617:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164d06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059626:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059631:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059638:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059643:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c37d7000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.059647:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059656:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059668:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:1.0:1788572549.059671:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.059677:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059684:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164cf8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059697:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059702:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059708:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059712:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74cda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.059714:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059719:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059724:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0xe44515a3 (ffff9627d4b57100) 00010000:00010000:1.0:1788572549.059726:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.059729:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059733:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164cff lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059763:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3327800/0x85ace67feb164cf1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a85 expref: 232 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059775:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb14d800/0x85ace67feb164d06 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a85 expref: 231 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.059782:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x95:0x0].0x0 (ffff9627c3d4b100) 00010000:00010000:1.0:1788572549.059784:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572549.059786:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb14d800, rc 0) 00000100:00100000:1.0:1788572549.059845:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+231:35867:x1875453360580864:12345-192.168.202.40@tcp:101:diff.0 Request processed in 896us (1094us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.059856:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4758 00000100:00100000:2.0:1788572549.064920:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.065102:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360580992 00000100:00100000:0.0:1788572549.065112:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360580992 00000100:00100000:0.0:1788572549.065171:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1362 00000100:00100000:0.0:1788572549.065191:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+52:2349:x1875453360580992:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.065211:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.065235:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dbcfa372/0x85ace67feb164d0d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a8c expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572549.065261:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x2a:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:0.0:1788572549.065280:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2a:0x0].0x0 since=897s v=(0 0) 00010000:00010000:0.0:1788572549.065284:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572549.065294:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dbcfa372/0x85ace67feb164d0d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a8c expref: 53 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.065311:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dbcfa372/0x85ace67feb164d0d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a8c expref: 53 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.065325:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92a00, rc 0) 00000100:00100000:0.0:1788572549.065393:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:2349:x1875453360580992:12345-192.168.202.40@tcp:101:diff.0 Request processed in 213us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.065401:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1362 00000100:00100000:2.0:1788572549.075829:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.075952:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360581248 00000100:00100000:3.0:1788572549.075962:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360581248 00000100:00100000:3.0:1788572549.076063:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4759 00000100:00100000:3.0:1788572549.076074:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+231:35867:x1875453360581248:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.076098:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.076120:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164d14 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a93 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.076248:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d1b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x95:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.076264:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.076278:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572549.076289:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x95:0x0]: returning size 3 00010000:00010000:3.0:1788572549.076424:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164d14 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a93 expref: 234 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.076460:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a93 expref: 233 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.076469:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x95:0x0].0x0 (ffff9627c3d4b100) 00010000:00010000:3.0:1788572549.076472:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572549.076475:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d600, rc 0) 00000100:00100000:3.0:1788572549.076534:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+233:35867:x1875453360581248:12345-192.168.202.40@tcp:101:diff.0 Request processed in 462us (715us total) trans 4294970873 rc 0/0 00000100:00100000:3.0:1788572549.076540:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4759 00000100:00100000:3.0:1788572549.081529:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.081578:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.081666:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360581376 00000100:00100000:2.0:1788572549.081679:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360581376 00002000:00100000:2.0:1788572549.081733:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x2a:0x0] (0->4095) req@ffff9626c2735500 x1875453360581376/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.081745:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2735500 with x1875453360581376 ext(0->4095) 00002000:00010000:2.0:1788572549.081751:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2735500 00000100:00100000:2.0:1788572549.081784:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 317 00000100:00100000:2.0:1788572549.081794:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:2349:x1875453360581376:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572549.082099:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:2349:x1875453360581376:12345-192.168.202.40@tcp:3:diff.0 Request processed in 305us (580us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572549.082108:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 317 00002000:00100000:2.0:1788572549.082115:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x2a:0x0] (0->4095) req@ffff9626c2735500 x1875453360581376/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.082129:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2735500 with x1875453360581376 ext(0->4095) 00002000:00010000:2.0:1788572549.082132:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2735500 00000100:00100000:1.0:1788572549.082211:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360581504 00000100:00100000:1.0:1788572549.082216:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360581504 00000100:00100000:1.0:1788572549.082264:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 335 00000100:00100000:1.0:1788572549.082272:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+233:2348:x1875453360581504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572549.082284:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572549.082310:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d1b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x95:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a93 expref: 233 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.082333:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x95:0x0].0x0 (ffff9627c3d4b100) 00010000:00010000:1.0:1788572549.082338:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572549.082363:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+232:2348:x1875453360581504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 92us (784us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.082370:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 335 00000100:00100000:3.0:1788572549.090619:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.090711:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360581760 00000100:00100000:0.0:1788572549.090722:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360581760 00000100:00100000:0.0:1788572549.090779:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1005 00000100:00100000:0.0:1788572549.090788:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+232:35867:x1875453360581760:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572549.091380:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+233:35867:x1875453360581760:12345-192.168.202.40@tcp:35:diff.0 Request processed in 590us (768us total) trans 4294970874 rc 0/0 00000100:00100000:0.0:1788572549.091390:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1005 00000100:00100000:0.0:1788572549.103668:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.109263:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360581888 00000100:00100000:2.0:1788572549.109282:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360581888 00000100:00100000:3.0:1788572549.113246:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4760 00000100:00100000:3.0:1788572549.113261:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+233:35867:x1875453360581888:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.113287:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.113308:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164d29 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0a9a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113378:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113393:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113406:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113420:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113438:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113446:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113742:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb164d3e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x94:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113751:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb164d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113758:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb164d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572549.113795:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x94:0x0]: returning size 3 00000004:00200000:3.0:1788572549.113800:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572549.113819:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb164d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113832:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb164d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113840:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113851:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113858:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113864:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572549.113868:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113875:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.113888:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572549.113891:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572549.113895:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116066:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d30 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116093:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116101:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116108:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116114:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572549.116119:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116127:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116142:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x2c9ecea3 (ffff9627c5716d00) 00010000:00010000:3.0:1788572549.116145:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572549.116150:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116156:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d37 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116211:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164d29 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a9a expref: 235 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116232:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d200/0x85ace67feb164d3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0a9a expref: 234 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.116240:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x94:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:3.0:1788572549.116242:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572549.116245:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d200, rc 0) 00000100:00100000:3.0:1788572549.116338:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+234:35867:x1875453360581888:12345-192.168.202.40@tcp:101:diff.0 Request processed in 3078us (12678us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.116347:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4760 00000100:00100000:2.0:1788572549.129766:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.129865:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360582016 00000100:00100000:1.0:1788572549.129878:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360582016 00000100:00100000:1.0:1788572549.129968:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1363 00000100:00100000:1.0:1788572549.129979:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+63:2350:x1875453360582016:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.129999:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:0.0:1788572549.130114:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1788572549.130342:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e81e9b24/0x85ace67feb164d45 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x29:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0aa1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572549.130373:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x29:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:1.0:1788572549.130394:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x29:0x0].0x0 since=897s v=(0 0) 00010000:00010000:1.0:1788572549.130399:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x29:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572549.130410:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e81e9b24/0x85ace67feb164d45 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x29:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aa1 expref: 64 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.130440:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e81e9b24/0x85ace67feb164d45 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x29:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aa1 expref: 64 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.130453:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3327800, rc 0) 00000100:00100000:1.0:1788572549.130517:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+64:2350:x1875453360582016:12345-192.168.202.40@tcp:101:diff.0 Request processed in 539us (765us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.130525:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1363 00000100:00100000:2.0:1788572549.136621:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.136725:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360582272 00000100:00100000:1.0:1788572549.136738:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360582272 00000100:00100000:1.0:1788572549.136802:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4761 00000100:00100000:1.0:1788572549.136813:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+234:35867:x1875453360582272:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.136832:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.136855:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164d4c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0aa8 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.145855:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164d53 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x94:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.145880:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164d53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.145893:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164d53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.145924:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x94:0x0]: returning size 3 00010000:00010000:2.0:1788572549.146093:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164d4c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aa8 expref: 237 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.146114:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164d53 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aa8 expref: 236 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.146123:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x94:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572549.146126:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.146129:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478f200, rc 0) 00000100:00100000:2.0:1788572549.146205:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+236:35867:x1875453360582272:12345-192.168.202.40@tcp:101:diff.0 Request processed in 9393us (9595us total) trans 4294970875 rc 0/0 00000100:00100000:2.0:1788572549.146214:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4761 00000100:00100000:0.0:1788572549.149449:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.149533:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360582400 00000100:00100000:2.0:1788572549.149544:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360582400 00000100:00100000:2.0:1788572549.149625:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 336 00000100:00100000:2.0:1788572549.149635:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+236:2349:x1875453360582400:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.149652:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.149678:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478f200/0x85ace67feb164d53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x94:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aa8 expref: 236 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.149698:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x94:0x0].0x0 (ffff9627c5716a00) 00010000:00010000:2.0:1788572549.149703:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.149768:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+235:2349:x1875453360582400:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 134us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.149775:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 336 00000100:00100000:0.0:1788572549.151510:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.151593:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360582528 00000100:00100000:1.0:1788572549.151604:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360582528 00002000:00100000:1.0:1788572549.151654:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x29:0x0] (0->4095) req@ffff9626c4921f80 x1875453360582528/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.151665:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4921f80 with x1875453360582528 ext(0->4095) 00002000:00010000:1.0:1788572549.151672:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4921f80 00000100:00100000:1.0:1788572549.151706:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 318 00000100:00100000:1.0:1788572549.151716:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+64:2350:x1875453360582528:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.151979:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+64:2350:x1875453360582528:12345-192.168.202.40@tcp:3:diff.0 Request processed in 262us (476us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.151986:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 318 00002000:00100000:1.0:1788572549.151993:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x29:0x0] (0->4095) req@ffff9626c4921f80 x1875453360582528/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.152000:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4921f80 with x1875453360582528 ext(0->4095) 00002000:00010000:1.0:1788572549.152040:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4921f80 00000100:00100000:0.0:1788572549.163247:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.163497:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360582784 00000100:00100000:1.0:1788572549.163508:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360582784 00000100:00100000:1.0:1788572549.163570:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1006 00000100:00100000:1.0:1788572549.163581:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+235:35867:x1875453360582784:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.164310:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+236:35867:x1875453360582784:12345-192.168.202.40@tcp:35:diff.0 Request processed in 727us (1069us total) trans 4294970876 rc 0/0 00000100:00100000:1.0:1788572549.164319:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1006 00000100:00100000:2.0:1788572549.172508:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.177090:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360582912 00000100:00100000:3.0:1788572549.177107:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360582912 00000100:00100000:3.0:1788572549.177172:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4762 00000100:00100000:3.0:1788572549.177182:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+236:35867:x1875453360582912:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.177209:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.177232:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164d61 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0aaf expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177304:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177322:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177335:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177350:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177355:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177361:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177684:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb164d76 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x93:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177693:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb164d76 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177701:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb164d76 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572549.177736:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x93:0x0]: returning size 3 00000004:00200000:3.0:1788572549.177741:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572549.177761:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb164d76 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177773:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb164d76 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177781:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177788:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177795:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177801:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572549.177805:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177813:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177825:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572549.177827:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572549.177831:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177837:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164d68 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177853:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177863:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177870:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177874:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572549.177878:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177884:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.177891:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x74f887a3 (ffff9627c5716d00) 00010000:00010000:3.0:1788572549.177893:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572549.177896:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.179119:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164d6f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.179180:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094c600/0x85ace67feb164d61 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aaf expref: 238 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.179202:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e76eba00/0x85ace67feb164d76 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aaf expref: 237 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.179210:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x93:0x0].0x0 (ffff9627c5716100) 00010000:00010000:3.0:1788572549.179213:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572549.179217:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e76eba00, rc 0) 00000100:00100000:3.0:1788572549.179359:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+237:35867:x1875453360582912:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2176us (6860us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.179370:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4762 00000100:00100000:0.0:1788572549.187495:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.187577:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360583040 00000100:00100000:3.0:1788572549.187592:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360583040 00000100:00100000:3.0:1788572549.187655:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1364 00000100:00100000:3.0:1788572549.187666:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+64:2349:x1875453360583040:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.187684:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.187708:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000059e23ee6/0x85ace67feb164d7d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x28:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ab6 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572549.187739:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x28:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:3.0:1788572549.187761:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x28:0x0].0x0 since=897s v=(0 0) 00010000:00010000:3.0:1788572549.187765:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x28:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572549.187774:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000059e23ee6/0x85ace67feb164d7d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x28:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ab6 expref: 65 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.187792:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000059e23ee6/0x85ace67feb164d7d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x28:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ab6 expref: 65 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.187805:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e76ea200, rc 0) 00000100:00100000:3.0:1788572549.187866:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2349:x1875453360583040:12345-192.168.202.40@tcp:101:diff.0 Request processed in 202us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.187873:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1364 00000100:00100000:0.0:1788572549.194454:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.195103:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360583296 00000100:00100000:2.0:1788572549.195115:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360583296 00000100:00100000:2.0:1788572549.195184:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4763 00000100:00100000:2.0:1788572549.195196:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+237:35867:x1875453360583296:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.195219:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.195243:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164d84 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0abd expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.195360:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164d8b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x93:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.195375:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164d8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.195391:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164d8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.195403:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x93:0x0]: returning size 3 00010000:00010000:2.0:1788572549.195543:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478fe00/0x85ace67feb164d84 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0abd expref: 240 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.195563:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164d8b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0abd expref: 239 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.195573:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x93:0x0].0x0 (ffff9627c5716100) 00010000:00010000:2.0:1788572549.195576:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.195579:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478ea00, rc 0) 00000100:00100000:2.0:1788572549.195722:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:35867:x1875453360583296:12345-192.168.202.40@tcp:101:diff.0 Request processed in 528us (1279us total) trans 4294970877 rc 0/0 00000100:00100000:2.0:1788572549.195729:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4763 00000100:00100000:2.0:1788572549.200604:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.200662:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.200738:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360583424 00000100:00100000:1.0:1788572549.200751:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360583424 00002000:00100000:1.0:1788572549.200805:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x28:0x0] (0->4095) req@ffff9626c13be680 x1875453360583424/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.200818:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13be680 with x1875453360583424 ext(0->4095) 00002000:00010000:1.0:1788572549.200825:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13be680 00000100:00100000:1.0:1788572549.200860:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 319 00000100:00100000:1.0:1788572549.200871:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2350:x1875453360583424:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.201158:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2350:x1875453360583424:12345-192.168.202.40@tcp:3:diff.0 Request processed in 286us (493us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.201169:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 319 00002000:00100000:1.0:1788572549.201176:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x28:0x0] (0->4095) req@ffff9626c13be680 x1875453360583424/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.201197:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13be680 with x1875453360583424 ext(0->4095) 00002000:00010000:1.0:1788572549.201201:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13be680 00000100:00100000:0.0:1788572549.201765:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360583552 00000100:00100000:0.0:1788572549.201771:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360583552 00000100:00100000:0.0:1788572549.201814:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 337 00000100:00100000:0.0:1788572549.201822:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2eff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:2350:x1875453360583552:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572549.201834:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572549.201857:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ea00/0x85ace67feb164d8b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x93:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0abd expref: 239 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.201875:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x93:0x0].0x0 (ffff9627c5716100) 00010000:00010000:0.0:1788572549.201881:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572549.201928:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2eff850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:2350:x1875453360583552:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 106us (1334us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.201934:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 337 00000100:00100000:2.0:1788572549.208261:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.208438:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360583808 00000100:00100000:0.0:1788572549.208450:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360583808 00000100:00100000:0.0:1788572549.208518:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1007 00000100:00100000:0.0:1788572549.208529:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:35867:x1875453360583808:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572549.208960:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:35867:x1875453360583808:12345-192.168.202.40@tcp:35:diff.0 Request processed in 432us (710us total) trans 4294970878 rc 0/0 00000100:00100000:0.0:1788572549.208968:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1007 00000100:00100000:2.0:1788572549.222116:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.223152:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360583936 00000100:00100000:0.0:1788572549.223166:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360583936 00000100:00100000:0.0:1788572549.223458:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4764 00000100:00100000:0.0:1788572549.223470:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:35867:x1875453360583936:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.223497:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.223519:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb164d99 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ac4 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.223594:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.223611:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.223622:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.223640:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.223647:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.223654:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.224174:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb164dae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x92:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.224987:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb164dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.224999:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb164dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.225072:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x92:0x0]: returning size 3 00000004:00200000:0.0:1788572549.225077:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.225099:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb164dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225113:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb164dae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225122:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225128:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225139:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225145:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.225149:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225157:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225170:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.225173:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.225177:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225265:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164da0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225280:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225287:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225292:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225297:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.225300:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225311:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225319:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0xbd5240a3 (ffff9627c29dad00) 00010000:00010000:0.0:1788572549.225321:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.225324:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225330:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164da7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225368:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb164d99 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ac4 expref: 241 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225382:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93400/0x85ace67feb164dae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ac4 expref: 240 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.225389:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x92:0x0].0x0 (ffff9627c29da300) 00010000:00010000:0.0:1788572549.225391:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.225393:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93400, rc 0) 00000100:00100000:0.0:1788572549.225545:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:35867:x1875453360583936:12345-192.168.202.40@tcp:101:diff.0 Request processed in 2076us (3449us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.225554:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4764 00000100:00100000:1.0:1788572549.235862:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.236160:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360584064 00000100:00100000:0.0:1788572549.236171:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360584064 00000100:00100000:0.0:1788572549.236234:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1365 00000100:00100000:0.0:1788572549.236245:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2349:x1875453360584064:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.236268:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.236290:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b98c08e0/0x85ace67feb164db5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x2b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0acb expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572549.236319:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x2b:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:0.0:1788572549.236340:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2b:0x0].0x0 since=897s v=(0 0) 00010000:00010000:0.0:1788572549.236345:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572549.236356:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b98c08e0/0x85ace67feb164db5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0acb expref: 66 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.236376:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b98c08e0/0x85ace67feb164db5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0acb expref: 66 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.236389:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93200, rc 0) 00000100:00100000:0.0:1788572549.236467:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2349:x1875453360584064:12345-192.168.202.40@tcp:101:diff.0 Request processed in 224us (617us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.236475:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1365 00000100:00100000:2.0:1788572549.239551:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.239638:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360584320 00000100:00100000:0.0:1788572549.239648:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360584320 00000100:00100000:0.0:1788572549.239714:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4765 00000100:00100000:0.0:1788572549.239726:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:35867:x1875453360584320:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.239748:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.239769:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb164dbc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ad2 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.239882:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164dc3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x92:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.239908:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164dc3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.239921:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164dc3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.239933:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x92:0x0]: returning size 3 00010000:00010000:0.0:1788572549.240106:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb164dbc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ad2 expref: 243 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.240125:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164dc3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ad2 expref: 242 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.240133:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x92:0x0].0x0 (ffff9627c29da300) 00010000:00010000:0.0:1788572549.240136:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.240139:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93e00, rc 0) 00000100:00100000:0.0:1788572549.240206:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:35867:x1875453360584320:12345-192.168.202.40@tcp:101:diff.0 Request processed in 481us (665us total) trans 4294970879 rc 0/0 00000100:00100000:0.0:1788572549.240214:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4765 00010000:00010000:3.0:1788572549.248198:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572549.256737:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.256831:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360584448 00000100:00100000:1.0:1788572549.256840:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360584448 00002000:00100000:1.0:1788572549.256894:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x2b:0x0] (0->4095) req@ffff9626c2734000 x1875453360584448/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.256914:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2734000 with x1875453360584448 ext(0->4095) 00002000:00010000:1.0:1788572549.256920:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2734000 00000100:00100000:1.0:1788572549.256956:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 320 00000100:00100000:1.0:1788572549.256966:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2350:x1875453360584448:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.257257:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2350:x1875453360584448:12345-192.168.202.40@tcp:3:diff.0 Request processed in 291us (529us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.257266:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 320 00002000:00100000:1.0:1788572549.257273:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x2b:0x0] (0->4095) req@ffff9626c2734000 x1875453360584448/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.257281:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2734000 with x1875453360584448 ext(0->4095) 00002000:00010000:1.0:1788572549.257284:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2734000 00000100:00100000:1.0:1788572549.265456:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.265679:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360584576 00000100:00100000:2.0:1788572549.265690:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360584576 00000100:00100000:2.0:1788572549.265771:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 338 00000100:00100000:2.0:1788572549.265782:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:2350:x1875453360584576:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.265800:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.265827:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164dc3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x92:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ad2 expref: 242 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.265847:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x92:0x0].0x0 (ffff9627c29da300) 00010000:00010000:2.0:1788572549.265853:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.265936:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2350:x1875453360584576:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 156us (491us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.265944:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 338 00000100:00100000:2.0:1788572549.271514:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.271609:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360584832 00000100:00100000:1.0:1788572549.271619:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360584832 00000100:00100000:1.0:1788572549.271672:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1008 00000100:00100000:1.0:1788572549.271682:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec039450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:35867:x1875453360584832:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.272189:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec039450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:35867:x1875453360584832:12345-192.168.202.40@tcp:35:diff.0 Request processed in 498us (675us total) trans 4294970880 rc 0/0 00000100:00100000:1.0:1788572549.272198:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1008 00000100:00100000:3.0:1788572549.277767:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.277885:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360584960 00000100:00100000:0.0:1788572549.277896:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360584960 00000100:00100000:0.0:1788572549.277972:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4766 00000100:00100000:0.0:1788572549.277983:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:35867:x1875453360584960:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.278051:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.278077:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb164dd1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ad9 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278163:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278192:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278204:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278219:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278226:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278232:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278549:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164de6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x91:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278558:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164de6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278565:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164de6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.278599:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x91:0x0]: returning size 3 00000004:00200000:0.0:1788572549.278604:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.278627:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164de6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278644:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164de6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278652:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278658:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278665:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278671:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.278675:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278683:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278694:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.278697:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.278701:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278708:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb164dd8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278720:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278726:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278732:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278737:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.278740:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278746:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278753:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x5abf9a3 (ffff9627c29dad00) 00010000:00010000:0.0:1788572549.278755:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.278758:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278763:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb164ddf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278795:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb164dd1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ad9 expref: 244 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278808:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92c00/0x85ace67feb164de6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ad9 expref: 243 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.278815:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x91:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572549.278817:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.278820:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92c00, rc 0) 00000100:00100000:0.0:1788572549.278898:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:35867:x1875453360584960:12345-192.168.202.40@tcp:101:diff.0 Request processed in 917us (1141us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.278910:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4766 00000100:00100000:3.0:1788572549.287729:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.287824:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585088 00000100:00100000:0.0:1788572549.287835:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585088 00000100:00100000:0.0:1788572549.287900:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 370 00000100:00100000:0.0:1788572549.287919:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:2348:x1875453360585088:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572549.287960:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.287981:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164ded lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ae0 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288084:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288100:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288111:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288126:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288132:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288139:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288478:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164e02 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x90:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288493:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164e02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288500:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164e02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.288524:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x90:0x0]: returning size 3 00000004:00200000:0.0:1788572549.288528:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.288556:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164e02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288569:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164e02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288578:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288584:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288591:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288597:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.288601:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288609:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288621:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.288624:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.288630:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288636:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93600/0x85ace67feb164df4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288649:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288654:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288660:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288665:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a92600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.288667:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288673:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288680:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x4e05b2a3 (ffff9627e7451e00) 00010000:00010000:0.0:1788572549.288683:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.288686:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288691:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb164dfb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288721:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92800/0x85ace67feb164ded lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ae0 expref: 245 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288735:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93000/0x85ace67feb164e02 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ae0 expref: 244 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288742:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x90:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572549.288744:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.288746:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93000, rc 0) 00010000:00010000:0.0:1788572549.288752:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.288760:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164e09 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ae7 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288784:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288792:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288799:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288810:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288817:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288824:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.288996:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164e1e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289039:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164e1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289047:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164e1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.289062:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8f:0x0]: returning size 3 00000004:00200000:0.0:1788572549.289065:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.289076:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164e1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289084:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164e1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289090:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289096:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289102:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289107:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.289110:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289115:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289123:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.289126:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.289128:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289133:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb164e10 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289144:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289150:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289156:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289161:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.289164:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289169:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289176:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x965f6ba3 (ffff9627d0ee2300) 00010000:00010000:0.0:1788572549.289178:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.289181:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289186:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb164e17 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289207:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb164e09 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ae7 expref: 246 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289219:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030c00/0x85ace67feb164e1e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ae7 expref: 245 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289225:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8f:0x0].0x0 (ffff9627d0ee2200) 00010000:00010000:0.0:1788572549.289228:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.289230:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c0030c00, rc 0) 00010000:00010000:0.0:1788572549.289235:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.289245:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb164e25 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0aee expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289268:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289276:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289284:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289296:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289302:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289309:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289488:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb164e3a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289498:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb164e3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289505:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb164e3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.289520:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8e:0x0]: returning size 3 00000004:00200000:0.0:1788572549.289524:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.289535:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb164e3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289543:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb164e3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289549:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289554:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289560:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289565:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.289568:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289573:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289581:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.289584:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.289587:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289592:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb164e2c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289602:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289608:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289615:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289620:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea2a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.289622:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289628:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289636:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0xdeb924a3 (ffff9627d0ee2000) 00010000:00010000:0.0:1788572549.289638:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.289641:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289647:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb164e33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289669:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb164e25 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aee expref: 247 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289681:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1679800/0x85ace67feb164e3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0aee expref: 246 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289688:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8e:0x0].0x0 (ffff9627c2bec500) 00010000:00010000:0.0:1788572549.289690:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.289692:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1679800, rc 0) 00010000:00010000:0.0:1788572549.289697:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.289705:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb164e41 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0af5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289727:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289735:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289743:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289754:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289760:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289767:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289849:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289857:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289863:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289868:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f46000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.289871:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289877:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289884:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.289887:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.289890:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289895:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e48 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289903:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289909:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289914:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289919:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962801f47000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.289922:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289927:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289933:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0xc8545ca3 (ffff9627d1866400) 00010000:00010000:0.0:1788572549.289936:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.289939:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289944:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47000/0x85ace67feb164e4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289967:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb164e41 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0af5 expref: 247 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289977:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:0.0:1788572549.289979:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572549.289983:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb164e41 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0af5 expref: 247 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.289993:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678400, rc -2) 00000100:00100000:0.0:1788572549.290134:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:2348:x1875453360585088:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2216us (2415us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572549.290145:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 370 00000100:00100000:3.0:1788572549.294278:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.294360:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585216 00000100:00100000:2.0:1788572549.294370:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585216 00000100:00100000:2.0:1788572549.294426:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1366 00000100:00100000:2.0:1788572549.294446:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+53:2349:x1875453360585216:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.294464:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.294493:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066743e3c/0x85ace67feb164e56 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b11 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.294524:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x2d:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:2.0:1788572549.294543:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2d:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.294548:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572549.294557:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066743e3c/0x85ace67feb164e56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b11 expref: 54 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.294574:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000066743e3c/0x85ace67feb164e56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b11 expref: 54 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.294589:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478f200, rc 0) 00000100:00100000:2.0:1788572549.294648:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+54:2349:x1875453360585216:12345-192.168.202.40@tcp:101:diff.0 Request processed in 203us (378us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.294655:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1366 00000100:00100000:3.0:1788572549.301728:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.301788:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.301827:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.301924:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585344 00000100:00100000:2.0:1788572549.301933:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585344 00000100:00100000:2.0:1788572549.301987:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585472 00000100:00100000:2.0:1788572549.301990:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585472 00000100:00100000:2.0:1788572549.302034:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585600 00000100:00100000:2.0:1788572549.302038:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585600 00000100:00100000:2.0:1788572549.302061:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1367 00000100:00100000:2.0:1788572549.302069:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+55:2350:x1875453360585344:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:2.0:1788572549.302088:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.302112:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007fb054ab/0x85ace67feb164e5d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b18 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.302142:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x2b:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:2.0:1788572549.302163:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2b:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.302168:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572549.302178:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007fb054ab/0x85ace67feb164e5d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b18 expref: 56 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.302195:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007fb054ab/0x85ace67feb164e5d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b18 expref: 56 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.302208:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478e400, rc 0) 00000100:00100000:2.0:1788572549.302266:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2350:x1875453360585344:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 197us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.302277:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1367 00000100:00100000:2.0:1788572549.302298:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1368 00000100:00100000:2.0:1788572549.302304:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2350:x1875453360585472:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:2.0:1788572549.302312:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.302324:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000874c0739/0x85ace67feb164e64 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x2a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b1f expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.302340:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x2a:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:2.0:1788572549.302349:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2a:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.302352:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572549.302357:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000874c0739/0x85ace67feb164e64 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b1f expref: 67 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.302369:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000874c0739/0x85ace67feb164e64 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b1f expref: 67 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.302378:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478ea00, rc 0) 00000100:00100000:2.0:1788572549.302399:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2350:x1875453360585472:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 95us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.302405:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1368 00000100:00100000:2.0:1788572549.302419:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1369 00000100:00100000:2.0:1788572549.302425:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+55:2350:x1875453360585600:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:2.0:1788572549.302443:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.302453:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020fbdab5/0x85ace67feb164e6b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x29:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b26 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.302465:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x29:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:2.0:1788572549.302473:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x29:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.302477:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x29:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572549.302481:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020fbdab5/0x85ace67feb164e6b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x29:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b26 expref: 56 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.302491:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000020fbdab5/0x85ace67feb164e6b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x29:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b26 expref: 56 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.302499:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c478fe00, rc 0) 00000100:00100000:2.0:1788572549.302516:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2350:x1875453360585600:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 92us (689us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.302521:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1369 00000100:00100000:3.0:1788572549.309859:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.310043:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585856 00000100:00100000:2.0:1788572549.310054:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585856 00000100:00100000:2.0:1788572549.310109:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4767 00000100:00100000:2.0:1788572549.310118:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:35867:x1875453360585856:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.310139:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.310162:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164e72 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b2d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.310269:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164e79 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x91:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.310284:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164e79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.310298:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164e79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.310310:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x91:0x0]: returning size 3 00010000:00010000:2.0:1788572549.310484:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164e72 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b2d expref: 249 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.310505:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164e79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b2d expref: 248 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.310514:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x91:0x0].0x0 (ffff9627e7451700) 00010000:00010000:2.0:1788572549.310517:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.310521:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2badc00, rc 0) 00000100:00100000:2.0:1788572549.310580:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:35867:x1875453360585856:12345-192.168.202.40@tcp:101:diff.0 Request processed in 462us (730us total) trans 4294970881 rc 0/0 00000100:00100000:2.0:1788572549.310588:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4767 00000040:00080000:0.0:1788572549.311408:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572549.311416:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:2.0:1788572549.317467:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.317580:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360585984 00000100:00100000:0.0:1788572549.317592:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360585984 00002000:00100000:0.0:1788572549.317652:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x2d:0x0] (0->4095) req@ffff9626c13be300 x1875453360585984/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572549.317662:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13be300 with x1875453360585984 ext(0->4095) 00002000:00010000:0.0:1788572549.317668:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c13be300 00000100:00100000:0.0:1788572549.317702:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 321 00000100:00100000:0.0:1788572549.317712:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2349:x1875453360585984:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572549.317991:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2349:x1875453360585984:12345-192.168.202.40@tcp:3:diff.0 Request processed in 279us (533us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572549.317999:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 321 00002000:00100000:0.0:1788572549.318040:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x2d:0x0] (0->4095) req@ffff9626c13be300 x1875453360585984/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572549.318050:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13be300 with x1875453360585984 ext(0->4095) 00002000:00010000:0.0:1788572549.318053:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c13be300 00000100:00100000:2.0:1788572549.318931:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.320050:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360586112 00000100:00100000:0.0:1788572549.320061:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360586112 00000100:00100000:0.0:1788572549.320122:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 339 00000100:00100000:0.0:1788572549.320132:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2347:x1875453360586112:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572549.320147:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572549.320169:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164e79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x91:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b2d expref: 248 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.320201:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x91:0x0].0x0 (ffff9627e7451700) 00010000:00010000:0.0:1788572549.320206:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572549.320262:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2347:x1875453360586112:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 131us (1333us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.320273:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 339 00000100:00100000:2.0:1788572549.331820:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.331919:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360586368 00000100:00100000:1.0:1788572549.331929:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360586368 00000100:00100000:1.0:1788572549.331988:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1009 00000100:00100000:1.0:1788572549.331999:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:35867:x1875453360586368:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.332452:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:35867:x1875453360586368:12345-192.168.202.40@tcp:35:diff.0 Request processed in 454us (642us total) trans 4294970882 rc 0/0 00000100:00100000:1.0:1788572549.332462:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1009 00000040:00080000:3.0:1788572549.338396:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 167 count 56 00000040:00080000:3.0:1788572549.338415:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1788572549.346299:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.346349:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.346455:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360586624 00000100:00100000:0.0:1788572549.346466:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360586624 00000100:00100000:0.0:1788572549.346525:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4768 00000100:00100000:0.0:1788572549.346535:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:35867:x1875453360586624:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.346560:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.346583:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e87 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b34 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.346686:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164e8e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x90:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.346699:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164e8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.346708:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164e8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.346718:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x90:0x0]: returning size 3 00010000:00010000:0.0:1788572549.346822:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164e87 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b34 expref: 251 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.346836:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164e8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b34 expref: 250 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.346842:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x90:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:0.0:1788572549.346844:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.346847:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47e00, rc 0) 00000100:00100000:0.0:1788572549.346967:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:35867:x1875453360586624:12345-192.168.202.40@tcp:101:diff.0 Request processed in 368us (611us total) trans 4294970883 rc 0/0 00000100:00100000:0.0:1788572549.346973:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4768 00000100:00100000:1.0:1788572549.347038:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360586752 00000100:00100000:1.0:1788572549.347042:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360586752 00000100:00100000:1.0:1788572549.347065:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 371 00000100:00100000:1.0:1788572549.347070:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:2350:x1875453360586752:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572549.347105:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.347116:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164e9c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b3b expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347153:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347163:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347172:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347182:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347187:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347191:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347327:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347334:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347340:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347346:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.347350:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347358:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347367:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:1.0:1788572549.347369:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.347372:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347377:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb164ea3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347383:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347386:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347390:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347393:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c37d7000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.347395:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347398:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347401:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x2f385ca3 (ffff9627c3e61900) 00010000:00010000:1.0:1788572549.347402:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.347404:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347407:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164eaa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347424:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164e9c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b3b expref: 251 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347450:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:1.0:1788572549.347452:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572549.347456:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164e9c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b3b expref: 251 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.347465:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc -2) 00000100:00100000:1.0:1788572549.347504:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:2350:x1875453360586752:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 433us (1154us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572549.347511:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 371 00000100:00100000:0.0:1788572549.354441:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.354764:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.355139:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360586880 00000100:00100000:2.0:1788572549.355149:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360586880 00002000:00100000:2.0:1788572549.355258:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x2b:0x0] (0->4095) req@ffff9626c12eaa00 x1875453360586880/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.355270:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12eaa00 with x1875453360586880 ext(0->4095) 00002000:00010000:2.0:1788572549.355277:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c12eaa00 00000100:00100000:2.0:1788572549.355357:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 322 00000100:00100000:2.0:1788572549.355367:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2347:x1875453360586880:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572549.355607:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2347:x1875453360586880:12345-192.168.202.40@tcp:3:diff.0 Request processed in 241us (1184us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572549.355615:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 322 00002000:00100000:2.0:1788572549.355622:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x2b:0x0] (0->4095) req@ffff9626c12eaa00 x1875453360586880/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.355630:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12eaa00 with x1875453360586880 ext(0->4095) 00002000:00010000:2.0:1788572549.355634:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c12eaa00 00000100:00100000:1.0:1788572549.356075:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360587008 00000100:00100000:1.0:1788572549.356082:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360587008 00000100:00100000:1.0:1788572549.356195:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 340 00000100:00100000:1.0:1788572549.356204:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:2348:x1875453360587008:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572549.356217:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572549.356244:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164e8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x90:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b34 expref: 250 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.356262:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x90:0x0].0x0 (ffff9627d0ee2c00) 00010000:00010000:1.0:1788572549.356267:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572549.356300:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:2348:x1875453360587008:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 98us (1537us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.356307:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 340 00000100:00100000:0.0:1788572549.359340:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.359419:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360587264 00000100:00100000:1.0:1788572549.359436:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360587264 00000100:00100000:1.0:1788572549.359493:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1010 00000100:00100000:1.0:1788572549.359505:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:35867:x1875453360587264:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.359950:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:35867:x1875453360587264:12345-192.168.202.40@tcp:35:diff.0 Request processed in 447us (620us total) trans 4294970884 rc 0/0 00000100:00100000:1.0:1788572549.359959:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1010 00000100:00100000:1.0:1788572549.366139:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.366214:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.366587:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360587520 00000100:00100000:3.0:1788572549.366598:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360587520 00000100:00100000:3.0:1788572549.366660:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4769 00000100:00100000:3.0:1788572549.366671:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:35867:x1875453360587520:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.366699:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.366723:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164eb1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b7a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.366834:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164eb8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.366849:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164eb8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.366861:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164eb8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572549.366873:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8f:0x0]: returning size 3 00010000:00010000:3.0:1788572549.367041:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164eb1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b7a expref: 253 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.367067:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164eb8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b7a expref: 252 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.367074:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8f:0x0].0x0 (ffff9627d0ee2200) 00010000:00010000:3.0:1788572549.367077:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572549.367081:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00000100:00100000:3.0:1788572549.367139:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:35867:x1875453360587520:12345-192.168.202.40@tcp:101:diff.0 Request processed in 470us (1011us total) trans 4294970885 rc 0/0 00000100:00100000:3.0:1788572549.367147:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4769 00000100:00100000:2.0:1788572549.367807:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360587648 00000100:00100000:2.0:1788572549.367815:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360587648 00000100:00100000:2.0:1788572549.367865:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 372 00000100:00100000:2.0:1788572549.367875:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:2347:x1875453360587648:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:0.0:1788572549.377163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.377216:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572549.377387:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.377411:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164ec6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0b81 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377490:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377504:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377515:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377534:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377540:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377545:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377720:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377727:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377733:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377738:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.377742:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377750:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377763:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572549.377766:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.377771:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377777:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ecd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377789:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377794:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377800:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377804:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b04a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.377806:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377812:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377818:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x1dd915a3 (ffff9627c3c32a00) 00010000:00010000:2.0:1788572549.377820:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.377823:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377828:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ed4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377857:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164ec6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b81 expref: 254 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377868:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572549.377870:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572549.377874:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164ec6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b81 expref: 254 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.377884:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc -2) 00000100:00100000:2.0:1788572549.377970:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:2347:x1875453360587648:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 10094us (11754us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572549.377979:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 372 00000100:00100000:3.0:1788572549.378169:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360587904 00000100:00100000:3.0:1788572549.378188:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360587904 00000100:00100000:3.0:1788572549.378245:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 341 00000100:00100000:3.0:1788572549.378255:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:2350:x1875453360587904:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572549.378266:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572549.378278:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164eb8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0b7a expref: 252 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.378292:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8f:0x0].0x0 (ffff9627d0ee2200) 00010000:00010000:3.0:1788572549.378296:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572549.378322:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:2350:x1875453360587904:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 69us (1106us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.378330:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 341 00000100:00100000:1.0:1788572549.378679:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360587776 00000100:00100000:1.0:1788572549.378685:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360587776 00002000:00100000:1.0:1788572549.378718:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x2a:0x0] (0->4095) req@ffff9626c12ec000 x1875453360587776/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.378728:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12ec000 with x1875453360587776 ext(0->4095) 00002000:00010000:1.0:1788572549.378733:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c12ec000 00000100:00100000:1.0:1788572549.378763:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 323 00000100:00100000:1.0:1788572549.378770:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360587776:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.378956:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360587776:12345-192.168.202.40@tcp:3:diff.0 Request processed in 186us (1801us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.378963:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 323 00002000:00100000:1.0:1788572549.378969:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x2a:0x0] (0->4095) req@ffff9626c12ec000 x1875453360587776/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.378977:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12ec000 with x1875453360587776 ext(0->4095) 00002000:00010000:1.0:1788572549.378981:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c12ec000 00000100:00100000:1.0:1788572549.390830:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.390944:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360588160 00000100:00100000:2.0:1788572549.390955:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360588160 00000100:00100000:2.0:1788572549.391054:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1011 00000100:00100000:2.0:1788572549.391066:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:35867:x1875453360588160:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572549.391509:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:35867:x1875453360588160:12345-192.168.202.40@tcp:35:diff.0 Request processed in 443us (687us total) trans 4294970886 rc 0/0 00000100:00100000:2.0:1788572549.391517:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1011 00000100:00100000:0.0:1788572549.402331:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.402717:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360588416 00000100:00100000:2.0:1788572549.402728:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360588416 00000100:00100000:2.0:1788572549.402796:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4770 00000100:00100000:2.0:1788572549.402806:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:35867:x1875453360588416:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.402832:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.402868:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164edb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0c76 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.403154:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ee2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.403169:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ee2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.403196:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ee2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.403209:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8e:0x0]: returning size 3 00010000:00010000:2.0:1788572549.403345:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164edb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0c76 expref: 255 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.403364:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ee2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0c76 expref: 254 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.403373:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8e:0x0].0x0 (ffff9627c2bec500) 00010000:00010000:2.0:1788572549.403376:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.403380:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:2.0:1788572549.403454:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:35867:x1875453360588416:12345-192.168.202.40@tcp:101:diff.0 Request processed in 649us (1136us total) trans 4294970887 rc 0/0 00000100:00100000:2.0:1788572549.403461:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4770 00000100:00100000:0.0:1788572549.409851:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.410035:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360588544 00000100:00100000:1.0:1788572549.410047:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360588544 00000100:00100000:1.0:1788572549.410110:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 373 00000100:00100000:1.0:1788572549.410121:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:2348:x1875453360588544:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:1.0:1788572549.430830:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.430877:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.430981:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360588800 00000100:00100000:2.0:1788572549.430991:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360588800 00000100:00100000:2.0:1788572549.431086:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 342 00000100:00100000:2.0:1788572549.431097:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:2349:x1875453360588800:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.431116:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.431142:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164ee2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0c76 expref: 256 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.431162:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8e:0x0].0x0 (ffff9627c2bec500) 00010000:00010000:2.0:1788572549.431168:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.431248:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:2349:x1875453360588800:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 152us (369us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.431255:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 342 00000100:00100000:0.0:1788572549.431992:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360588672 00000100:00100000:0.0:1788572549.431999:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360588672 00002000:00100000:0.0:1788572549.432065:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x29:0x0] (0->4095) req@ffff9627ff5ec700 x1875453360588672/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572549.432075:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff5ec700 with x1875453360588672 ext(0->4095) 00002000:00010000:0.0:1788572549.432081:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ff5ec700 00000100:00100000:0.0:1788572549.432160:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 324 00000100:00100000:0.0:1788572549.432169:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2349:x1875453360588672:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572549.433379:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2349:x1875453360588672:12345-192.168.202.40@tcp:3:diff.0 Request processed in 1203us (2551us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572549.433390:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 324 00002000:00100000:0.0:1788572549.433400:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x29:0x0] (0->4095) req@ffff9627ff5ec700 x1875453360588672/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572549.433410:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff5ec700 with x1875453360588672 ext(0->4095) 00002000:00010000:0.0:1788572549.433415:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627ff5ec700 00010000:00010000:2.0:1788572549.441381:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.441412:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ef0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0bf8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441491:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441506:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441517:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441533:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441540:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441547:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441736:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441744:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441751:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441757:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2badc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.441762:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441769:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441782:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572549.441785:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.441789:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441795:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164ef7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441808:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441813:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441819:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441823:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.441826:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441832:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441842:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0xa16187a3 (ffff9627c3c32a00) 00010000:00010000:2.0:1788572549.441845:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.441847:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441853:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164efe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441891:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ef0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0bf8 expref: 255 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441903:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572549.441905:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572549.441909:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164ef0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0bf8 expref: 255 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.441918:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05000, rc -2) 00000100:00100000:2.0:1788572549.442034:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+253:2348:x1875453360588544:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 31885us (32166us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572549.442044:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 373 00000100:00100000:1.0:1788572549.442371:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.442528:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360589056 00000100:00100000:1.0:1788572549.442537:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360589056 00000100:00100000:1.0:1788572549.442581:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1012 00000100:00100000:1.0:1788572549.442589:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+253:35867:x1875453360589056:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.442939:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:35867:x1875453360589056:12345-192.168.202.40@tcp:35:diff.0 Request processed in 349us (568us total) trans 4294970888 rc 0/0 00000100:00100000:1.0:1788572549.442947:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1012 00000100:00100000:1.0:1788572549.449819:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.450054:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360589184 00000100:00100000:0.0:1788572549.450064:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360589184 00000100:00100000:0.0:1788572549.450174:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4771 00000100:00100000:0.0:1788572549.450285:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:35867:x1875453360589184:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.450308:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.450334:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164f05 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ce6 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.450407:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164f0c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.450423:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164f0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.450447:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164f0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.450482:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00000004:00200000:0.0:1788572549.450485:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.450505:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164f0c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.450533:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb164f05 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ce6 expref: 256 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.450550:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f47e00/0x85ace67feb164f0c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ce6 expref: 255 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.450560:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572549.450563:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572549.450567:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47e00, rc 0) 00000100:00100000:0.0:1788572549.450664:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:35867:x1875453360589184:12345-192.168.202.40@tcp:101:diff.0 Request processed in 480us (856us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.450672:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4771 00000100:00100000:1.0:1788572549.456743:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.456838:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360589440 00000100:00100000:3.0:1788572549.456858:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360589440 00000100:00100000:3.0:1788572549.456926:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4772 00000100:00100000:3.0:1788572549.456938:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:35867:x1875453360589440:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.456958:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.456980:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f13 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ced expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572549.457110:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x81:0x0]: returning size 16384 00010000:00010000:3.0:1788572549.457252:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f13 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ced expref: 257 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.457266:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572549.457270:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572549.457275:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f13 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ced expref: 257 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.457293:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d600, rc 0) 00000100:00100000:3.0:1788572549.457351:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:35867:x1875453360589440:12345-192.168.202.40@tcp:101:diff.0 Request processed in 416us (619us total) trans 4294970889 rc 301/301 00000100:00100000:3.0:1788572549.457359:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4772 00000100:00100000:2.0:1788572549.467826:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.467932:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360589568 00000100:00100000:1.0:1788572549.467943:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360589568 00000100:00100000:1.0:1788572549.468042:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1013 00000100:00100000:1.0:1788572549.468052:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:35867:x1875453360589568:12345-192.168.202.40@tcp:37:diff.0 00000100:00100000:1.0:1788572549.471558:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:35867:x1875453360589568:12345-192.168.202.40@tcp:37:diff.0 Request processed in 3496us (3732us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.471572:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1013 00000100:00100000:3.0:1788572549.476990:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.477162:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360589824 00000100:00100000:1.0:1788572549.477173:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360589824 00000100:00100000:1.0:1788572549.477238:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1014 00000100:00100000:1.0:1788572549.477248:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec038050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:35867:x1875453360589824:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.477510:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec038050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+257:35867:x1875453360589824:12345-192.168.202.40@tcp:35:diff.0 Request processed in 262us (528us total) trans 4294970890 rc 0/0 00000100:00100000:1.0:1788572549.477519:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1014 00000100:00100000:2.0:1788572549.480876:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.481045:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360589952 00000100:00100000:1.0:1788572549.481057:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360589952 00000100:00100000:1.0:1788572549.481119:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4773 00000100:00100000:1.0:1788572549.481130:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+257:35867:x1875453360589952:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.481159:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.481191:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164f21 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0cf4 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481263:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481277:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481288:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481301:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481308:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481315:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481604:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164f36 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481616:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164f36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481623:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164f36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572549.481657:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8b:0x0]: returning size 3 00000004:00200000:1.0:1788572549.481662:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572549.481680:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164f36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481692:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164f36 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481700:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481706:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481713:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481719:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.481724:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481732:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481744:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572549.481747:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.481752:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481758:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481771:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481777:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481783:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481788:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23eee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.481791:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481796:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481802:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x9beb5ca3 (ffff9627c3e61900) 00010000:00010000:1.0:1788572549.481805:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.481807:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481812:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481842:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164f21 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0cf4 expref: 259 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481856:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74cda00/0x85ace67feb164f36 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0cf4 expref: 258 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.481864:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627c3e61d00) 00010000:00010000:1.0:1788572549.481866:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572549.481869:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74cda00, rc 0) 00000100:00100000:1.0:1788572549.481950:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+258:35867:x1875453360589952:12345-192.168.202.40@tcp:101:diff.0 Request processed in 822us (1083us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.481958:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4773 00000100:00100000:1.0:1788572549.486727:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.486825:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360590080 00000100:00100000:3.0:1788572549.486835:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360590080 00000100:00100000:3.0:1788572549.486892:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1370 00000100:00100000:3.0:1788572549.486910:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+56:2350:x1875453360590080:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.486929:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.486952:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb164f3d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0cfb expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572549.486986:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x47:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:3.0:1788572549.487041:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=897s v=(0 0) 00010000:00010000:3.0:1788572549.487046:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572549.487055:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb164f3d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0cfb expref: 57 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.487073:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000bbaa91e/0x85ace67feb164f3d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0cfb expref: 57 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.487086:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094c600, rc 0) 00000100:00100000:3.0:1788572549.487141:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2350:x1875453360590080:12345-192.168.202.40@tcp:101:diff.0 Request processed in 240us (425us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.487148:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1370 00000100:00100000:3.0:1788572549.497821:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.497926:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360590336 00000100:00100000:2.0:1788572549.497936:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360590336 00000100:00100000:2.0:1788572549.498228:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4774 00000100:00100000:2.0:1788572549.498240:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+258:35867:x1875453360590336:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.498262:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.498284:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb164f44 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d02 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.498387:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164f4b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.498403:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164f4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.498416:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164f4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.498440:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8b:0x0]: returning size 3 00010000:00010000:2.0:1788572549.498575:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90200/0x85ace67feb164f44 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d02 expref: 261 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.498592:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164f4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d02 expref: 260 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.498601:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627c3e61d00) 00010000:00010000:2.0:1788572549.498604:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.498608:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90c00, rc 0) 00000100:00100000:2.0:1788572549.498708:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:35867:x1875453360590336:12345-192.168.202.40@tcp:101:diff.0 Request processed in 470us (897us total) trans 4294970891 rc 0/0 00000100:00100000:2.0:1788572549.498716:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4774 00000100:00100000:0.0:1788572549.506340:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.506419:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360590464 00000100:00100000:1.0:1788572549.506439:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360590464 00000100:00100000:1.0:1788572549.506505:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 343 00000100:00100000:1.0:1788572549.506515:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:2349:x1875453360590464:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572549.506532:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572549.506555:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164f4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d02 expref: 260 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.506573:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8b:0x0].0x0 (ffff9627c3e61d00) 00010000:00010000:1.0:1788572549.506578:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572549.506635:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+259:2349:x1875453360590464:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 121us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.506642:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 343 00000100:00100000:0.0:1788572549.518710:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.518810:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360590592 00000100:00100000:2.0:1788572549.518820:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360590592 00002000:00100000:2.0:1788572549.518870:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x47:0x0] (0->4095) req@ffff9626c4921180 x1875453360590592/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.518889:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4921180 with x1875453360590592 ext(0->4095) 00002000:00010000:2.0:1788572549.518895:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4921180 00000100:00100000:2.0:1788572549.518938:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 325 00000100:00100000:2.0:1788572549.518948:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2349:x1875453360590592:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572549.519216:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2349:x1875453360590592:12345-192.168.202.40@tcp:3:diff.0 Request processed in 268us (514us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572549.519225:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 325 00002000:00100000:2.0:1788572549.519233:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x47:0x0] (0->4095) req@ffff9626c4921180 x1875453360590592/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.519240:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4921180 with x1875453360590592 ext(0->4095) 00002000:00010000:2.0:1788572549.519242:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4921180 00000100:00100000:0.0:1788572549.525675:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.525763:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360590848 00000100:00100000:1.0:1788572549.525775:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360590848 00000100:00100000:1.0:1788572549.525841:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1015 00000100:00100000:1.0:1788572549.525862:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+259:35867:x1875453360590848:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.526332:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:35867:x1875453360590848:12345-192.168.202.40@tcp:35:diff.0 Request processed in 480us (665us total) trans 4294970892 rc 0/0 00000100:00100000:1.0:1788572549.526341:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1015 00000100:00100000:0.0:1788572549.530951:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.531252:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360590976 00000100:00100000:1.0:1788572549.531263:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360590976 00000100:00100000:1.0:1788572549.531322:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4775 00000100:00100000:1.0:1788572549.531333:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:35867:x1875453360590976:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.531358:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.531380:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164f59 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d09 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.531468:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.531484:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.531495:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.531510:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.531517:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.531523:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534352:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164f6e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534375:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164f6e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534388:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164f6e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572549.534447:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x82:0x0]: returning size 3 00000004:00200000:1.0:1788572549.534452:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572549.534479:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164f6e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534493:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164f6e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534503:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534510:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534518:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534523:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23eee00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.534528:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534536:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534551:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572549.534554:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.534559:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534566:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23eee00/0x85ace67feb164f60 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534580:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534586:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534592:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534598:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.534600:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534607:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534614:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x6e9b5ca3 (ffff9627c3e61900) 00010000:00010000:1.0:1788572549.534616:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.534619:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534625:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb164f67 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534671:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb164f59 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d09 expref: 262 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534685:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05000/0x85ace67feb164f6e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d09 expref: 261 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.534693:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3e61300) 00010000:00010000:1.0:1788572549.534695:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572549.534697:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05000, rc 0) 00000100:00100000:1.0:1788572549.534771:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+261:35867:x1875453360590976:12345-192.168.202.40@tcp:101:diff.0 Request processed in 3439us (3828us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.534779:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4775 00000100:00100000:0.0:1788572549.545084:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.545132:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360591104 00000100:00100000:2.0:1788572549.545142:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360591104 00000100:00100000:2.0:1788572549.545205:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1371 00000100:00100000:2.0:1788572549.545216:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360591104:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.545234:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.545259:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000050ae67b6/0x85ace67feb164f75 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d10 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.545291:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4b:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:2.0:1788572549.545313:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.545318:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572549.545330:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000050ae67b6/0x85ace67feb164f75 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d10 expref: 68 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.545355:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000050ae67b6/0x85ace67feb164f75 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d10 expref: 68 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.545369:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90200, rc 0) 00000100:00100000:2.0:1788572549.545439:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2349:x1875453360591104:12345-192.168.202.40@tcp:101:diff.0 Request processed in 225us (364us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.545447:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1371 00000100:00100000:0.0:1788572549.555441:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.557064:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360591360 00000100:00100000:1.0:1788572549.557078:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360591360 00000100:00100000:1.0:1788572549.557162:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4776 00000100:00100000:1.0:1788572549.557174:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+261:35867:x1875453360591360:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.557196:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.557219:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164f7c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d17 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.557334:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164f83 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.557349:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164f83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.557363:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164f83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572549.557376:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x82:0x0]: returning size 3 00010000:00010000:1.0:1788572549.557527:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b04a00/0x85ace67feb164f7c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d17 expref: 264 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.557548:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164f83 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d17 expref: 263 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.557558:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3e61300) 00010000:00010000:1.0:1788572549.557561:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572549.557565:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:1.0:1788572549.557627:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:35867:x1875453360591360:12345-192.168.202.40@tcp:101:diff.0 Request processed in 456us (2208us total) trans 4294970893 rc 0/0 00000100:00100000:1.0:1788572549.557635:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4776 00000100:00100000:0.0:1788572549.575630:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.575723:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360591488 00000100:00100000:2.0:1788572549.575736:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360591488 00002000:00100000:2.0:1788572549.575789:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4b:0x0] (0->4095) req@ffff9626c1b5bb80 x1875453360591488/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.575801:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b5bb80 with x1875453360591488 ext(0->4095) 00002000:00010000:2.0:1788572549.575807:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1b5bb80 00000100:00100000:2.0:1788572549.575831:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 326 00000100:00100000:2.0:1788572549.575840:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2349:x1875453360591488:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572549.576298:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2349:x1875453360591488:12345-192.168.202.40@tcp:3:diff.0 Request processed in 456us (677us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572549.576308:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 326 00002000:00100000:2.0:1788572549.576315:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4b:0x0] (0->4095) req@ffff9626c1b5bb80 x1875453360591488/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.576322:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b5bb80 with x1875453360591488 ext(0->4095) 00002000:00010000:2.0:1788572549.576326:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1b5bb80 00000100:00100000:0.0:1788572549.577128:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.577879:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360591616 00000100:00100000:2.0:1788572549.577888:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360591616 00000100:00100000:2.0:1788572549.577956:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 344 00000100:00100000:2.0:1788572549.577965:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:2348:x1875453360591616:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.577980:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.578028:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb164f83 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x82:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d17 expref: 263 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.578047:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x82:0x0].0x0 (ffff9627c3e61300) 00010000:00010000:2.0:1788572549.578052:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.578261:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+262:2348:x1875453360591616:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 296us (1134us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.578270:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 344 00000100:00100000:0.0:1788572549.579719:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.579795:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360591872 00000100:00100000:1.0:1788572549.579803:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360591872 00000100:00100000:1.0:1788572549.579854:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1016 00000100:00100000:1.0:1788572549.579863:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+262:35867:x1875453360591872:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.580308:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:35867:x1875453360591872:12345-192.168.202.40@tcp:35:diff.0 Request processed in 446us (594us total) trans 4294970894 rc 0/0 00000100:00100000:1.0:1788572549.580317:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1016 00000100:00100000:0.0:1788572549.588336:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.588476:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360592000 00000100:00100000:3.0:1788572549.588492:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360592000 00000100:00100000:3.0:1788572549.588576:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4777 00000100:00100000:3.0:1788572549.588590:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:35867:x1875453360592000:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.588625:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.588657:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164f91 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d1e expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.588758:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.588781:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.588796:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.588821:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.588829:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.588837:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595508:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164fa6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595533:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595546:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.595593:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8a:0x0]: returning size 3 00000004:00200000:2.0:1788572549.595598:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572549.595620:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595633:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164fa6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595642:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595650:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595659:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595666:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.595671:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595679:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595693:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.595696:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.595702:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595709:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164f98 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595727:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595733:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595739:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595744:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.595747:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595752:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595758:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xe44515a3 (ffff9627d599a700) 00010000:00010000:2.0:1788572549.595760:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.595763:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595768:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb164f9f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595811:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164f91 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d1e expref: 265 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595826:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7b90c00/0x85ace67feb164fa6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d1e expref: 264 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.595833:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d20cba00) 00010000:00010000:2.0:1788572549.595835:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.595837:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7b90c00, rc 0) 00000100:00100000:2.0:1788572549.598155:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+264:35867:x1875453360592000:12345-192.168.202.40@tcp:101:diff.0 Request processed in 9557us (9822us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.598169:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4777 00000100:00100000:2.0:1788572549.603514:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.603597:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360592128 00000100:00100000:1.0:1788572549.603609:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360592128 00000100:00100000:1.0:1788572549.603673:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1372 00000100:00100000:1.0:1788572549.603684:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+57:2349:x1875453360592128:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.603703:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.603724:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1d26361/0x85ace67feb164fad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d25 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572549.603751:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x48:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:1.0:1788572549.603771:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=897s v=(0 0) 00010000:00010000:1.0:1788572549.603775:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572549.603784:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1d26361/0x85ace67feb164fad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d25 expref: 58 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.603802:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1d26361/0x85ace67feb164fad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d25 expref: 58 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.603815:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b04a00, rc 0) 00000100:00100000:1.0:1788572549.603879:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2349:x1875453360592128:12345-192.168.202.40@tcp:101:diff.0 Request processed in 196us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.603886:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1372 00000100:00100000:0.0:1788572549.614510:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.614617:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360592384 00000100:00100000:3.0:1788572549.614629:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360592384 00000100:00100000:3.0:1788572549.614944:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4778 00000100:00100000:3.0:1788572549.614955:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+264:35867:x1875453360592384:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572549.614977:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.614997:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb164fb4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d2c expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.615148:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb164fbb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.615164:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb164fbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.615177:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb164fbb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572549.615189:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8a:0x0]: returning size 3 00010000:00010000:3.0:1788572549.615321:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53200/0x85ace67feb164fb4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d2c expref: 267 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.615341:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb164fbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d2c expref: 266 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.615350:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d20cba00) 00010000:00010000:3.0:1788572549.615353:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572549.615357:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53600, rc 0) 00000100:00100000:3.0:1788572549.615435:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+266:35867:x1875453360592384:12345-192.168.202.40@tcp:101:diff.0 Request processed in 473us (926us total) trans 4294970895 rc 0/0 00000100:00100000:3.0:1788572549.615443:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4778 00000100:00100000:0.0:1788572549.623190:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.623231:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.623340:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360592512 00000100:00100000:1.0:1788572549.623350:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360592512 00002000:00100000:1.0:1788572549.623401:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x48:0x0] (0->4095) req@ffff9627e786e680 x1875453360592512/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.623412:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786e680 with x1875453360592512 ext(0->4095) 00002000:00010000:1.0:1788572549.623418:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786e680 00000100:00100000:1.0:1788572549.623464:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 327 00000100:00100000:1.0:1788572549.623474:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2347:x1875453360592512:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.623716:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2347:x1875453360592512:12345-192.168.202.40@tcp:3:diff.0 Request processed in 243us (535us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.623727:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 327 00002000:00100000:1.0:1788572549.623734:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x48:0x0] (0->4095) req@ffff9627e786e680 x1875453360592512/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.623741:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786e680 with x1875453360592512 ext(0->4095) 00002000:00010000:1.0:1788572549.623745:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786e680 00000100:00100000:2.0:1788572549.630174:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360592640 00000100:00100000:2.0:1788572549.630192:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360592640 00000100:00100000:2.0:1788572549.630280:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 345 00000100:00100000:2.0:1788572549.630291:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+266:2348:x1875453360592640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.630309:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.630335:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb164fbb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d2c expref: 266 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.630357:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8a:0x0].0x0 (ffff9627d20cba00) 00010000:00010000:2.0:1788572549.630362:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.630418:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+265:2348:x1875453360592640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 128us (7185us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.630439:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 345 00000100:00100000:1.0:1788572549.633235:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.633400:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360592896 00000100:00100000:2.0:1788572549.633410:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360592896 00000100:00100000:2.0:1788572549.633468:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1017 00000100:00100000:2.0:1788572549.633479:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+265:35867:x1875453360592896:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572549.633914:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+266:35867:x1875453360592896:12345-192.168.202.40@tcp:35:diff.0 Request processed in 435us (687us total) trans 4294970896 rc 0/0 00000100:00100000:2.0:1788572549.633923:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1017 00000100:00100000:1.0:1788572549.637248:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.637388:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360593024 00000100:00100000:2.0:1788572549.637398:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360593024 00000100:00100000:2.0:1788572549.637467:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4779 00000100:00100000:2.0:1788572549.637478:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+266:35867:x1875453360593024:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.637503:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.637526:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164fc9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d33 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637598:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637614:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637626:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637642:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637650:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637656:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637983:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164fde lrc: 3/1,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.637992:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164fde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638001:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164fde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.638068:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x89:0x0]: returning size 3 00000004:00200000:2.0:1788572549.638072:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572549.638105:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164fde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638119:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164fde lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638132:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638139:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638146:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638151:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094dc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.638155:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638163:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638175:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.638179:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.638183:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638190:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb164fd0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638206:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638213:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638218:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638224:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.638227:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638234:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638242:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x2c9ecea3 (ffff9627d20cb300) 00010000:00010000:2.0:1788572549.638244:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.638247:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638254:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb164fd7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638286:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb164fc9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d33 expref: 268 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638299:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2badc00/0x85ace67feb164fde lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d33 expref: 267 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.638307:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627d20cb600) 00010000:00010000:2.0:1788572549.638309:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.638311:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2badc00, rc 0) 00000100:00100000:2.0:1788572549.638369:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+267:35867:x1875453360593024:12345-192.168.202.40@tcp:101:diff.0 Request processed in 893us (1132us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.638377:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4779 00000100:00100000:3.0:1788572549.644532:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.644611:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360593152 00000100:00100000:2.0:1788572549.644623:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360593152 00000100:00100000:2.0:1788572549.644685:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1373 00000100:00100000:2.0:1788572549.644697:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+58:2347:x1875453360593152:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.644716:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.644744:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb5107f8/0x85ace67feb164fe5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d3a expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.644776:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4a:0x0] initial LVB size: 3, mtime: 0x6a9b7367, atime: 0x6a9b7367, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:2.0:1788572549.644798:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.644804:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572549.644814:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb5107f8/0x85ace67feb164fe5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d3a expref: 59 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.644833:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb5107f8/0x85ace67feb164fe5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d3a expref: 59 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.644847:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050c00, rc 0) 00000100:00100000:2.0:1788572549.644931:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+59:2347:x1875453360593152:12345-192.168.202.40@tcp:101:diff.0 Request processed in 237us (409us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.644939:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1373 00000100:00100000:1.0:1788572549.660662:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.660771:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360593408 00000100:00100000:2.0:1788572549.660784:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360593408 00000100:00100000:2.0:1788572549.660860:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4780 00000100:00100000:2.0:1788572549.660871:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+267:35867:x1875453360593408:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.660893:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.660927:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb164fec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d41 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.661072:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb164ff3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.661088:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb164ff3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.661100:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb164ff3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.661112:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x89:0x0]: returning size 3 00010000:00010000:2.0:1788572549.661246:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb164fec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d41 expref: 270 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.661266:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb164ff3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d41 expref: 269 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.661274:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627d20cb600) 00010000:00010000:2.0:1788572549.661277:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.661281:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc 0) 00000100:00100000:2.0:1788572549.661471:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:35867:x1875453360593408:12345-192.168.202.40@tcp:101:diff.0 Request processed in 601us (820us total) trans 4294970897 rc 0/0 00000100:00100000:2.0:1788572549.661479:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4780 00000100:00100000:3.0:1788572549.669546:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.669652:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.672186:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360593536 00000100:00100000:1.0:1788572549.672201:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360593536 00002000:00100000:1.0:1788572549.672266:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4a:0x0] (0->4095) req@ffff9626c36c5180 x1875453360593536/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.672276:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c5180 with x1875453360593536 ext(0->4095) 00002000:00010000:1.0:1788572549.672282:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36c5180 00000100:00100000:1.0:1788572549.672322:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 328 00000100:00100000:1.0:1788572549.672330:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+59:2348:x1875453360593536:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.672599:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+59:2348:x1875453360593536:12345-192.168.202.40@tcp:3:diff.0 Request processed in 269us (2946us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.672607:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 328 00002000:00100000:1.0:1788572549.672614:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4a:0x0] (0->4095) req@ffff9626c36c5180 x1875453360593536/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.672621:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c5180 with x1875453360593536 ext(0->4095) 00002000:00010000:1.0:1788572549.672625:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36c5180 00000100:00100000:0.0:1788572549.674792:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360593664 00000100:00100000:0.0:1788572549.674805:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360593664 00000100:00100000:0.0:1788572549.674866:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 346 00000100:00100000:0.0:1788572549.674875:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:2349:x1875453360593664:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572549.674890:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572549.675322:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb164ff3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x89:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d41 expref: 269 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.675342:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x89:0x0].0x0 (ffff9627d20cb600) 00010000:00010000:0.0:1788572549.675348:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572549.675448:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+268:2349:x1875453360593664:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 574us (5914us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.675457:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 346 00000100:00100000:3.0:1788572549.687123:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.687219:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360593920 00000100:00100000:0.0:1788572549.687230:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360593920 00000100:00100000:0.0:1788572549.687296:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1018 00000100:00100000:0.0:1788572549.687307:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+268:35867:x1875453360593920:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572549.687748:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:35867:x1875453360593920:12345-192.168.202.40@tcp:35:diff.0 Request processed in 442us (633us total) trans 4294970898 rc 0/0 00000100:00100000:0.0:1788572549.687756:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1018 00000100:00100000:3.0:1788572549.693413:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.693519:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360594048 00000100:00100000:1.0:1788572549.693530:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360594048 00000100:00100000:1.0:1788572549.693594:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4781 00000100:00100000:1.0:1788572549.693604:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:35867:x1875453360594048:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.693631:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.693655:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb165001 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d48 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.693728:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.693744:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.693756:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.693774:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.693781:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.693788:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694413:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb165016 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694442:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb165016 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694452:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb165016 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.694488:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x88:0x0]: returning size 3 00000004:00200000:0.0:1788572549.694493:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.694512:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb165016 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694524:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb165016 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694532:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694538:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694544:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694550:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c37d7000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.694554:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694562:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694574:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572549.694577:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.694581:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694588:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165008 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694601:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694607:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694612:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694617:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c23ee600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.694620:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694629:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694636:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x74f887a3 (ffff9627c3e61600) 00010000:00010000:0.0:1788572549.694639:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.694642:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694648:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb16500f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694683:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb165001 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d48 expref: 271 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694697:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962801f46000/0x85ace67feb165016 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d48 expref: 270 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.694704:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1866400) 00010000:00010000:0.0:1788572549.694707:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.694709:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f46000, rc 0) 00000100:00100000:0.0:1788572549.694780:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+270:35867:x1875453360594048:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1177us (1376us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.694788:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4781 00000100:00100000:3.0:1788572549.708676:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.709229:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360594176 00000100:00100000:0.0:1788572549.709240:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360594176 00000100:00100000:0.0:1788572549.709302:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1374 00000100:00100000:0.0:1788572549.709313:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2350:x1875453360594176:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.709336:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.709358:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb16501d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d4f expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572549.709391:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4a:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:0.0:1788572549.709416:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=897s v=(0 0) 00010000:00010000:0.0:1788572549.709420:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572549.709441:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb16501d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d4f expref: 69 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.709458:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002431b4f8/0x85ace67feb16501d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d4f expref: 69 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.709472:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962801f47000, rc 0) 00000100:00100000:0.0:1788572549.709540:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+69:2350:x1875453360594176:12345-192.168.202.40@tcp:101:diff.0 Request processed in 229us (876us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.709547:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1374 00000100:00100000:1.0:1788572549.714307:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.714481:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360594432 00000100:00100000:2.0:1788572549.714492:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360594432 00000100:00100000:2.0:1788572549.714547:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4782 00000100:00100000:2.0:1788572549.714558:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+270:35867:x1875453360594432:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.714579:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.714598:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165024 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d56 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.714704:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16502b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.714717:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16502b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.714728:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16502b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.714738:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x88:0x0]: returning size 3 00010000:00010000:2.0:1788572549.714863:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165024 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d56 expref: 273 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.714882:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16502b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d56 expref: 272 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.714891:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1866400) 00010000:00010000:2.0:1788572549.714962:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.714969:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050400, rc 0) 00000100:00100000:2.0:1788572549.717684:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:35867:x1875453360594432:12345-192.168.202.40@tcp:101:diff.0 Request processed in 3113us (3373us total) trans 4294970899 rc 0/0 00000100:00100000:2.0:1788572549.717698:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4782 00000100:00100000:3.0:1788572549.727280:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.727417:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360594560 00000100:00100000:1.0:1788572549.727439:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360594560 00002000:00100000:1.0:1788572549.727495:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4a:0x0] (0->4095) req@ffff9626c36c6d80 x1875453360594560/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.727508:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c6d80 with x1875453360594560 ext(0->4095) 00002000:00010000:1.0:1788572549.727515:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c36c6d80 00000100:00100000:1.0:1788572549.727550:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 329 00000100:00100000:1.0:1788572549.727559:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+69:2348:x1875453360594560:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.727815:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+69:2348:x1875453360594560:12345-192.168.202.40@tcp:3:diff.0 Request processed in 256us (544us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.727823:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 329 00002000:00100000:1.0:1788572549.727831:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4a:0x0] (0->4095) req@ffff9626c36c6d80 x1875453360594560/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.727839:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c6d80 with x1875453360594560 ext(0->4095) 00002000:00010000:1.0:1788572549.727842:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c36c6d80 00000100:00100000:3.0:1788572549.735642:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.735742:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360594688 00000100:00100000:2.0:1788572549.735752:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360594688 00000100:00100000:2.0:1788572549.735822:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 347 00000100:00100000:2.0:1788572549.735833:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:2349:x1875453360594688:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.735853:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.735877:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16502b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x88:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d56 expref: 272 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.735897:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x88:0x0].0x0 (ffff9627d1866400) 00010000:00010000:2.0:1788572549.735902:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.735958:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:2349:x1875453360594688:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 127us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.735966:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 347 00000100:00100000:3.0:1788572549.741354:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.741454:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360594944 00000100:00100000:1.0:1788572549.741464:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360594944 00000100:00100000:1.0:1788572549.741527:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1019 00000100:00100000:1.0:1788572549.741537:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:35867:x1875453360594944:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.741992:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:35867:x1875453360594944:12345-192.168.202.40@tcp:35:diff.0 Request processed in 454us (646us total) trans 4294970900 rc 0/0 00000100:00100000:1.0:1788572549.742000:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1019 00000100:00100000:3.0:1788572549.754162:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.754261:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360595072 00000100:00100000:0.0:1788572549.754271:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360595072 00000100:00100000:0.0:1788572549.754342:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4783 00000100:00100000:0.0:1788572549.754352:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:35867:x1875453360595072:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.754376:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.754400:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb165039 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d5d expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754485:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754501:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754513:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754530:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754537:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754544:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754812:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb16504e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754820:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb16504e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754826:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb16504e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.754858:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x87:0x0]: returning size 3 00000004:00200000:0.0:1788572549.754863:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.754881:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb16504e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754892:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb16504e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754913:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754919:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754925:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754931:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c37d7000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.754935:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754942:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754954:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572549.754957:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.754962:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754967:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb165040 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754978:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754984:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754989:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.754995:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c1678400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.754997:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.755047:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.755057:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xbd5240a3 (ffff9627d1866e00) 00010000:00010000:0.0:1788572549.755059:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.755062:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.755068:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb165047 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.755102:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb165039 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d5d expref: 274 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.755115:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3800/0x85ace67feb16504e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d5d expref: 273 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.755123:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627d1866700) 00010000:00010000:0.0:1788572549.755125:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.755127:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea3800, rc 0) 00000100:00100000:0.0:1788572549.755193:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+273:35867:x1875453360595072:12345-192.168.202.40@tcp:101:diff.0 Request processed in 842us (1043us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.755200:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4783 00000100:00100000:3.0:1788572549.768903:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.769061:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360595200 00000100:00100000:1.0:1788572549.769072:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360595200 00000100:00100000:1.0:1788572549.769145:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1375 00000100:00100000:1.0:1788572549.769155:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+69:2348:x1875453360595200:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.769174:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.769196:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000018940c/0x85ace67feb165055 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d64 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572549.769233:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x46:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736d, blocks: 0xb 00010000:00010000:1.0:1788572549.769252:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=897s v=(0 0) 00010000:00010000:1.0:1788572549.769257:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x46:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572549.769267:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000018940c/0x85ace67feb165055 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d64 expref: 70 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.769286:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000018940c/0x85ace67feb165055 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d64 expref: 70 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.769300:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23eee00, rc 0) 00000100:00100000:1.0:1788572549.769476:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+70:2348:x1875453360595200:12345-192.168.202.40@tcp:101:diff.0 Request processed in 322us (594us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.769483:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1375 00000100:00100000:3.0:1788572549.780373:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.780482:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360595456 00000100:00100000:2.0:1788572549.780492:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360595456 00000100:00100000:2.0:1788572549.780553:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4784 00000100:00100000:2.0:1788572549.780564:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+273:35867:x1875453360595456:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.780582:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.780603:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16505c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d6b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.780708:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165063 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.780722:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165063 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.780734:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165063 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.780744:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x87:0x0]: returning size 3 00010000:00010000:2.0:1788572549.780872:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16505c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d6b expref: 276 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.780892:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165063 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d6b expref: 275 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.780914:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627d1866700) 00010000:00010000:2.0:1788572549.780917:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.780921:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051600, rc 0) 00000100:00100000:2.0:1788572549.780994:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+275:35867:x1875453360595456:12345-192.168.202.40@tcp:101:diff.0 Request processed in 432us (630us total) trans 4294970901 rc 0/0 00000100:00100000:2.0:1788572549.781048:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4784 00000100:00100000:3.0:1788572549.794645:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.794694:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.794784:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360595584 00000100:00100000:1.0:1788572549.794794:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360595584 00002000:00100000:1.0:1788572549.794843:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x46:0x0] (0->4095) req@ffff9626c36c7480 x1875453360595584/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.794855:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c7480 with x1875453360595584 ext(0->4095) 00002000:00010000:1.0:1788572549.794860:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c36c7480 00000100:00100000:1.0:1788572549.794898:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 330 00000100:00100000:1.0:1788572549.794915:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+70:2348:x1875453360595584:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572549.795001:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360595712 00000100:00100000:0.0:1788572549.795027:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360595712 00000100:00100000:0.0:1788572549.795066:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 348 00000100:00100000:0.0:1788572549.795073:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+275:2350:x1875453360595712:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572549.795085:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572549.795125:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165063 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x87:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d6b expref: 275 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.795144:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x87:0x0].0x0 (ffff9627d1866700) 00010000:00010000:0.0:1788572549.795150:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572549.795199:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+70:2348:x1875453360595584:12345-192.168.202.40@tcp:3:diff.0 Request processed in 284us (562us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.795211:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 330 00002000:00100000:1.0:1788572549.795218:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x46:0x0] (0->4095) req@ffff9626c36c7480 x1875453360595584/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.795226:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c7480 with x1875453360595584 ext(0->4095) 00002000:00010000:1.0:1788572549.795230:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c36c7480 00000100:00100000:0.0:1788572549.802442:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+274:2350:x1875453360595712:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 7345us (7723us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.807771:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 348 00000100:00100000:1.0:1788572549.813140:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.813237:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360595968 00000100:00100000:2.0:1788572549.813248:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360595968 00000100:00100000:2.0:1788572549.813309:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1020 00000100:00100000:2.0:1788572549.813320:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+274:35867:x1875453360595968:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572549.813759:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+275:35867:x1875453360595968:12345-192.168.202.40@tcp:35:diff.0 Request processed in 440us (628us total) trans 4294970902 rc 0/0 00000100:00100000:2.0:1788572549.813767:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1020 00000100:00100000:3.0:1788572549.816994:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.817132:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596096 00000100:00100000:2.0:1788572549.817143:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596096 00000100:00100000:2.0:1788572549.817211:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4785 00000100:00100000:2.0:1788572549.817222:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+275:35867:x1875453360596096:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.817246:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.817268:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165071 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d72 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817339:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817355:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817365:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817380:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817387:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817394:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817745:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb165086 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817755:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb165086 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817763:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb165086 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.817799:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x86:0x0]: returning size 3 00000004:00200000:2.0:1788572549.817803:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572549.817823:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb165086 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817835:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb165086 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817847:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817853:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817861:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817866:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.817872:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817880:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.817893:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.818052:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.818059:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818066:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165078 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818079:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818085:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818090:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818096:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.818099:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818105:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818112:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x5abf9a3 (ffff9627c613b400) 00010000:00010000:2.0:1788572549.818115:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.818117:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818123:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16507f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818157:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051600/0x85ace67feb165071 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d72 expref: 277 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818171:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051800/0x85ace67feb165086 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d72 expref: 276 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.818237:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627c613b900) 00010000:00010000:2.0:1788572549.818240:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.818242:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051800, rc 0) 00000100:00100000:2.0:1788572549.818308:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+276:35867:x1875453360596096:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1087us (1325us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.818316:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4785 00000100:00100000:1.0:1788572549.830083:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.830130:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.830209:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596352 00000100:00100000:0.0:1788572549.830220:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596352 00000100:00100000:0.0:1788572549.830287:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 374 00000100:00100000:0.0:1788572549.830298:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+276:2350:x1875453360596352:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:2.0:1788572549.830302:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596224 00000100:00100000:2.0:1788572549.830306:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596224 00010000:00010000:0.0:1788572549.830335:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1788572549.830338:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1376 00000100:00100000:2.0:1788572549.830345:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+70:2348:x1875453360596224:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.830354:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.830357:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16508d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d80 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.830368:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb165094 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d79 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572549.830392:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x47:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:2.0:1788572549.830407:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=897s v=(0 0) 00010000:00010000:2.0:1788572549.830412:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x47:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572549.830414:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.830421:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb165094 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d79 expref: 71 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830439:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.830450:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006a1ee4ef/0x85ace67feb165094 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x47:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d79 expref: 71 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830463:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.830464:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051600, rc 0) 00010000:00010000:0.0:1788572549.830483:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830489:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830495:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572549.830520:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2348:x1875453360596224:12345-192.168.202.40@tcp:101:diff.0 Request processed in 175us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.830526:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1376 00010000:00010000:0.0:1788572549.830819:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1650a9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830829:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1650a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830836:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1650a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.830855:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x85:0x0]: returning size 3 00000004:00200000:0.0:1788572549.830859:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572549.830879:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1650a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.830889:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1650a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831162:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831170:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831181:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831187:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.831191:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831200:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831212:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572549.831216:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.831220:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831227:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb16509b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831241:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831247:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831253:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831257:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.831260:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831270:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831278:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x4e05b2a3 (ffff9627d1866e00) 00010000:00010000:0.0:1788572549.831280:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.831284:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831290:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831322:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16508d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d80 expref: 278 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831333:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c1678400/0x85ace67feb1650a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d80 expref: 277 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831340:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:0.0:1788572549.831343:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.831345:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c1678400, rc 0) 00010000:00010000:0.0:1788572549.831352:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.831361:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1650b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d87 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831385:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831393:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831404:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831414:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831420:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.831438:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841239:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1650c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841262:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1650c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841277:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1650c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572549.841311:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x84:0x0]: returning size 3 00000004:00200000:1.0:1788572549.841317:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572549.841346:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1650c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841358:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1650c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841369:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841376:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841384:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841390:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0031c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.841395:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841403:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841417:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572549.841421:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.841438:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.841445:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0031c00/0x85ace67feb1650b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843174:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843190:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843198:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843203:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c0030e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.843208:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843218:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843232:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x965f6ba3 (ffff9627d0ee2300) 00010000:00010000:1.0:1788572549.843235:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.843239:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843245:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c0030e00/0x85ace67feb1650be lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843297:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c37d7000/0x85ace67feb1650b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d87 expref: 279 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843323:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c23ee600/0x85ace67feb1650c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d87 expref: 278 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.843331:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627c3e61700) 00010000:00010000:1.0:1788572549.843334:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572549.843336:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c23ee600, rc 0) 00010000:00010000:1.0:1788572549.843349:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.845532:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb1650cc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d8e expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.845612:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.845626:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.845637:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.845652:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.845658:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.845664:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846000:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1650e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846051:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1650e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846058:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1650e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.846080:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x83:0x0]: returning size 3 00000004:00200000:2.0:1788572549.846084:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572549.846105:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1650e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846117:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1650e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846125:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846135:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846142:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846149:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627c2bade00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.846153:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846160:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846172:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.846174:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.846179:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846185:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c2bade00/0x85ace67feb1650d3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846198:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846204:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846209:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846214:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96280094d600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.846218:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846224:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846231:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xdeb924a3 (ffff9627c613b100) 00010000:00010000:2.0:1788572549.846233:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.846236:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846241:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb1650da lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846274:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb1650cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d8e expref: 280 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846288:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094dc00/0x85ace67feb1650e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d8e expref: 279 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846295:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627c613b000) 00010000:00010000:2.0:1788572549.846297:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.846300:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094dc00, rc 0) 00010000:00010000:2.0:1788572549.846308:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.846317:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb1650e8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0d95 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846341:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846349:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846357:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846367:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846374:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846380:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846482:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846490:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846495:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846500:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.846503:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846509:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846517:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.846520:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.846522:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846528:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1650ef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846537:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846543:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846550:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846555:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.846557:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846563:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846569:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xc8545ca3 (ffff9627c613b800) 00010000:00010000:2.0:1788572549.846572:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.846575:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846581:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb1650f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846603:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb1650e8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d95 expref: 280 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846618:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.846620:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572549.846624:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb1650e8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0d95 expref: 280 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.846633:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715200, rc -2) 00000100:00100000:2.0:1788572549.846716:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+279:2350:x1875453360596352:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 16418us (16584us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572549.846726:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 374 00000100:00100000:2.0:1788572549.853837:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.853938:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596608 00000100:00100000:0.0:1788572549.853948:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596608 00000100:00100000:0.0:1788572549.854022:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4786 00000100:00100000:0.0:1788572549.854033:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+279:35867:x1875453360596608:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.854054:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.854074:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0db1 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.854188:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165104 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.854204:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165104 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.854215:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165104 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.854226:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x86:0x0]: returning size 3 00010000:00010000:0.0:1788572549.854352:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1650fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0db1 expref: 282 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.854371:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165104 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0db1 expref: 281 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.854380:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627c613b900) 00010000:00010000:0.0:1788572549.854383:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.854387:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea2a00, rc 0) 00000100:00100000:0.0:1788572549.854454:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+281:35867:x1875453360596608:12345-192.168.202.40@tcp:101:diff.0 Request processed in 423us (627us total) trans 4294970903 rc 0/0 00000100:00100000:0.0:1788572549.854462:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4786 00000100:00100000:1.0:1788572549.864594:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.864631:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.864709:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.864814:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596736 00000100:00100000:3.0:1788572549.864825:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596736 00000100:00100000:3.0:1788572549.864892:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596864 00000100:00100000:3.0:1788572549.864899:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596864 00000100:00100000:3.0:1788572549.864922:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360596992 00000100:00100000:3.0:1788572549.864925:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360596992 00000100:00100000:3.0:1788572549.864946:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1377 00000100:00100000:3.0:1788572549.864956:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+59:2348:x1875453360596736:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572549.864976:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.864998:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb165112 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0db8 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572549.865063:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x49:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:3.0:1788572549.865083:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=897s v=(0 0) 00010000:00010000:3.0:1788572549.865087:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572549.865097:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb165112 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0db8 expref: 60 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.865115:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000795c09f0/0x85ace67feb165112 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0db8 expref: 60 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.865129:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52000, rc 0) 00000100:00100000:3.0:1788572549.865182:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+60:2348:x1875453360596736:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 227us (597us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.865189:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1377 00000100:00100000:3.0:1788572549.865208:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1378 00000100:00100000:3.0:1788572549.865215:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2348:x1875453360596864:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572549.865223:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.865254:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb165119 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0dbf expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572549.865270:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x48:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:3.0:1788572549.865281:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=897s v=(0 0) 00010000:00010000:3.0:1788572549.865284:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x48:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572549.865290:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb165119 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dbf expref: 73 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.865302:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e1556e3/0x85ace67feb165119 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x48:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dbf expref: 73 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.865310:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53200, rc 0) 00000100:00100000:3.0:1788572549.865332:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360596864:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 118us (700us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.865338:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1378 00000100:00100000:3.0:1788572549.865352:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1379 00000100:00100000:3.0:1788572549.865359:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2348:x1875453360596992:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572549.865367:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572549.865376:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000702c2999/0x85ace67feb165120 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0dc6 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572549.865393:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x49:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:3.0:1788572549.865403:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=897s v=(0 0) 00010000:00010000:3.0:1788572549.865406:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x49:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572549.865411:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000702c2999/0x85ace67feb165120 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dc6 expref: 73 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.865421:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000702c2999/0x85ace67feb165120 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x49:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dc6 expref: 73 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572549.865443:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd52600, rc 0) 00000100:00100000:3.0:1788572549.865464:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360596992:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 105us (754us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572549.865470:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1379 00000100:00100000:3.0:1788572549.867088:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.867132:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.867217:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360597248 00000100:00100000:2.0:1788572549.867225:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360597248 00002000:00100000:2.0:1788572549.867269:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x47:0x0] (0->4095) req@ffff9627fb30c000 x1875453360597248/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.867279:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30c000 with x1875453360597248 ext(0->4095) 00002000:00010000:2.0:1788572549.867285:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30c000 00000100:00100000:2.0:1788572549.867317:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 331 00000100:00100000:2.0:1788572549.867326:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360597248:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572549.867573:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360597248:12345-192.168.202.40@tcp:3:diff.0 Request processed in 247us (439us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572549.867581:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 331 00002000:00100000:2.0:1788572549.867586:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x47:0x0] (0->4095) req@ffff9627fb30c000 x1875453360597248/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.867594:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30c000 with x1875453360597248 ext(0->4095) 00002000:00010000:2.0:1788572549.867597:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30c000 00000100:00100000:0.0:1788572549.867775:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360597120 00000100:00100000:0.0:1788572549.867780:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360597120 00000100:00100000:0.0:1788572549.867826:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 349 00000100:00100000:0.0:1788572549.867835:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+281:2349:x1875453360597120:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572549.867846:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572549.867868:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165104 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x86:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0db1 expref: 281 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.867885:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x86:0x0].0x0 (ffff9627c613b900) 00010000:00010000:0.0:1788572549.867891:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572549.867936:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:2349:x1875453360597120:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 103us (855us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.867943:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 349 00000100:00100000:3.0:1788572549.873318:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.873452:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360597504 00000100:00100000:1.0:1788572549.873463:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360597504 00000100:00100000:1.0:1788572549.873533:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1021 00000100:00100000:1.0:1788572549.873544:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:35867:x1875453360597504:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.874327:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+281:35867:x1875453360597504:12345-192.168.202.40@tcp:35:diff.0 Request processed in 781us (1015us total) trans 4294970904 rc 0/0 00000100:00100000:1.0:1788572549.874337:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1021 00000100:00100000:3.0:1788572549.883166:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.883466:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.883554:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360597760 00000100:00100000:2.0:1788572549.883564:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360597760 00000100:00100000:2.0:1788572549.883627:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4787 00000100:00100000:2.0:1788572549.883638:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+281:35867:x1875453360597760:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572549.883661:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.883682:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165127 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0dcd expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.883783:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16512e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.883797:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16512e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.883808:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16512e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572549.883819:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x85:0x0]: returning size 3 00010000:00010000:2.0:1788572549.883953:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165127 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dcd expref: 284 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.883970:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16512e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dcd expref: 283 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.883978:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:2.0:1788572549.883981:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572549.883985:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715600, rc 0) 00000100:00100000:2.0:1788572549.884076:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:35867:x1875453360597760:12345-192.168.202.40@tcp:101:diff.0 Request processed in 440us (920us total) trans 4294970905 rc 0/0 00000100:00100000:2.0:1788572549.884084:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4787 00000100:00100000:1.0:1788572549.891139:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360597888 00000100:00100000:1.0:1788572549.891157:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360597888 00000100:00100000:0.0:1788572549.891388:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 375 00000100:00100000:0.0:1788572549.891400:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:2349:x1875453360597888:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572549.891472:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.891497:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16513c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0dd4 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891560:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891575:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891588:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891606:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891614:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891622:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891817:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891826:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891832:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891838:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.891844:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891852:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891865:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572549.891868:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.891873:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891880:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165143 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891892:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891910:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891915:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891920:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572549.891923:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891928:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891936:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x2f385ca3 (ffff9627d0ee2300) 00010000:00010000:0.0:1788572549.891939:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572549.891942:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891948:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16514a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891979:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16513c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dd4 expref: 284 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.891989:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572549.891991:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572549.891994:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16513c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dd4 expref: 284 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.892022:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea2a00, rc -2) 00000100:00100000:0.0:1788572549.892105:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:2349:x1875453360597888:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 707us (8638us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572549.892114:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 375 00000100:00100000:3.0:1788572549.892677:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.892812:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.894122:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360598144 00000100:00100000:1.0:1788572549.894135:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360598144 00000100:00100000:1.0:1788572549.894225:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 350 00000100:00100000:1.0:1788572549.894236:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:2347:x1875453360598144:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572549.894252:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572549.894277:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16512e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x85:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0dcd expref: 283 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.894298:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x85:0x0].0x0 (ffff9627d0ee2000) 00010000:00010000:1.0:1788572549.894304:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572549.894358:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:2347:x1875453360598144:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 124us (1546us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572549.894365:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 350 00000100:00100000:1.0:1788572549.894405:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360598016 00000100:00100000:1.0:1788572549.894409:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360598016 00002000:00100000:1.0:1788572549.894451:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x49:0x0] (0->4095) req@ffff9627fb30d880 x1875453360598016/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.894463:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30d880 with x1875453360598016 ext(0->4095) 00002000:00010000:1.0:1788572549.894469:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30d880 00000100:00100000:1.0:1788572549.894496:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 332 00000100:00100000:1.0:1788572549.894504:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+60:2350:x1875453360598016:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.894721:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+60:2350:x1875453360598016:12345-192.168.202.40@tcp:3:diff.0 Request processed in 216us (2044us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.894728:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 332 00002000:00100000:1.0:1788572549.894734:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x49:0x0] (0->4095) req@ffff9627fb30d880 x1875453360598016/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.894742:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30d880 with x1875453360598016 ext(0->4095) 00002000:00010000:1.0:1788572549.894746:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30d880 00000100:00100000:3.0:1788572549.903450:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.903527:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360598400 00000100:00100000:0.0:1788572549.903539:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360598400 00000100:00100000:0.0:1788572549.903602:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1022 00000100:00100000:0.0:1788572549.903613:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:35867:x1875453360598400:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572549.904057:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:35867:x1875453360598400:12345-192.168.202.40@tcp:35:diff.0 Request processed in 445us (615us total) trans 4294970906 rc 0/0 00000100:00100000:0.0:1788572549.904065:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1022 00000100:00100000:0.0:1788572549.915123:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.915229:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.915326:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360598784 00000100:00100000:2.0:1788572549.915336:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360598784 00000100:00100000:2.0:1788572549.915396:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 376 00000100:00100000:2.0:1788572549.915407:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:2350:x1875453360598784:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:1.0:1788572549.915795:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360598656 00000100:00100000:1.0:1788572549.915800:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360598656 00000100:00100000:1.0:1788572549.915840:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4788 00000100:00100000:1.0:1788572549.915848:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:35867:x1875453360598656:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572549.915859:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.915879:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165151 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0e13 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.916025:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165158 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.916040:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165158 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.916052:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165158 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572549.916063:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x84:0x0]: returning size 3 00010000:00010000:1.0:1788572549.916204:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165151 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e13 expref: 288 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.916228:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165158 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e13 expref: 287 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.916237:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627c3e61700) 00010000:00010000:1.0:1788572549.916240:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572549.916244:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc 0) 00000100:00100000:1.0:1788572549.916283:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:35867:x1875453360598656:12345-192.168.202.40@tcp:101:diff.0 Request processed in 436us (1054us total) trans 4294970907 rc 0/0 00000100:00100000:1.0:1788572549.916290:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4788 00010000:00010000:2.0:1788572549.922941:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572549.922970:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165166 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0e1a expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923069:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923082:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923094:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923110:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923116:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923123:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923312:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923324:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923330:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923335:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.923339:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923346:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923358:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.923360:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.923365:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923371:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb16516d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923381:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923385:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923390:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923395:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572549.923398:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923403:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923408:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x1dd915a3 (ffff9627c613b800) 00010000:00010000:2.0:1788572549.923411:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572549.923413:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923418:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165174 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923482:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165166 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e1a expref: 287 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923493:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572549.923495:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572549.923498:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165166 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e1a expref: 287 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.923508:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3714000, rc -2) 00000100:00100000:2.0:1788572549.923585:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:2350:x1875453360598784:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 8178us (8471us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572549.923595:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 376 00000100:00100000:3.0:1788572549.926916:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.926962:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.927057:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360599040 00000100:00100000:0.0:1788572549.927066:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360599040 00000100:00100000:0.0:1788572549.927121:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 351 00000100:00100000:0.0:1788572549.927131:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:2349:x1875453360599040:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572549.927146:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572549.927168:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165158 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x84:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e13 expref: 285 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.927194:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x84:0x0].0x0 (ffff9627c3e61700) 00010000:00010000:0.0:1788572549.927199:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572549.927249:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+284:2349:x1875453360599040:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 119us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572549.927256:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 351 00000100:00100000:1.0:1788572549.927508:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360598912 00000100:00100000:1.0:1788572549.927513:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360598912 00002000:00100000:1.0:1788572549.927546:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x48:0x0] (0->4095) req@ffff9627fb30c380 x1875453360598912/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.927557:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30c380 with x1875453360598912 ext(0->4095) 00002000:00010000:1.0:1788572549.927561:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30c380 00000100:00100000:1.0:1788572549.927587:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 333 00000100:00100000:1.0:1788572549.927594:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360598912:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572549.927795:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360598912:12345-192.168.202.40@tcp:3:diff.0 Request processed in 199us (831us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572549.927802:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 333 00002000:00100000:1.0:1788572549.927807:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x48:0x0] (0->4095) req@ffff9627fb30c380 x1875453360598912/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572549.927815:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30c380 with x1875453360598912 ext(0->4095) 00002000:00010000:1.0:1788572549.927819:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb30c380 00000100:00100000:2.0:1788572549.955188:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572549.955370:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360599296 00000100:00100000:1.0:1788572549.955379:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360599296 00000100:00100000:1.0:1788572549.955447:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1023 00000100:00100000:1.0:1788572549.955458:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+284:35867:x1875453360599296:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572549.955890:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:35867:x1875453360599296:12345-192.168.202.40@tcp:35:diff.0 Request processed in 432us (720us total) trans 4294970908 rc 0/0 00000100:00100000:1.0:1788572549.955907:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1023 00000100:00100000:3.0:1788572549.961466:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572549.961515:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572549.961718:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360599552 00000100:00100000:0.0:1788572549.961730:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360599552 00000100:00100000:0.0:1788572549.961795:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4789 00000100:00100000:0.0:1788572549.961807:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:35867:x1875453360599552:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572549.961832:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572549.961853:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16517b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0e91 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.961972:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165182 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.961989:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165182 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.962032:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165182 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572549.962048:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x83:0x0]: returning size 3 00010000:00010000:0.0:1788572549.962175:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb16517b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e91 expref: 288 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.962194:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165182 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e91 expref: 287 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572549.962202:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627c613b000) 00010000:00010000:0.0:1788572549.962205:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572549.962208:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea2a00, rc 0) 00000100:00100000:0.0:1788572549.962265:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:35867:x1875453360599552:12345-192.168.202.40@tcp:101:diff.0 Request processed in 460us (808us total) trans 4294970909 rc 0/0 00000100:00100000:0.0:1788572549.962272:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4789 00000100:00100000:1.0:1788572549.962844:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360599680 00000100:00100000:1.0:1788572549.962851:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360599680 00000100:00100000:1.0:1788572549.962889:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 377 00000100:00100000:1.0:1788572549.962905:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:2349:x1875453360599680:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572549.967679:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572549.967708:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165190 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0e98 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967765:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967779:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967789:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967803:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967810:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967818:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.967999:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968043:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968050:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968056:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.968061:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968068:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968080:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572549.968083:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.968087:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968093:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165197 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968104:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968110:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968116:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968121:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572549.968124:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968130:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968136:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xa16187a3 (ffff9627c3e61e00) 00010000:00010000:1.0:1788572549.968138:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572549.968140:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968145:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16519e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968175:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165190 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e98 expref: 289 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968294:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:1.0:1788572549.968296:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572549.968301:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165190 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e98 expref: 289 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572549.968311:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050400, rc -2) 00000100:00100000:1.0:1788572549.968442:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:2349:x1875453360599680:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 5545us (6925us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572549.968455:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 377 00000100:00100000:3.0:1788572549.978439:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.979470:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360599808 00000100:00100000:2.0:1788572549.979484:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360599808 00002000:00100000:2.0:1788572549.979541:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x49:0x0] (0->4095) req@ffff9627ca8c8700 x1875453360599808/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/0 e 0 to 0 dl 1788572560 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.979553:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8c8700 with x1875453360599808 ext(0->4095) 00002000:00010000:2.0:1788572549.979560:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ca8c8700 00000100:00100000:2.0:1788572549.979599:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 334 00000100:00100000:2.0:1788572549.979610:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360599808:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572549.979878:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360599808:12345-192.168.202.40@tcp:3:diff.0 Request processed in 268us (1459us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572549.979886:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 334 00002000:00100000:2.0:1788572549.980075:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x49:0x0] (0->4095) req@ffff9627ca8c8700 x1875453360599808/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:210/0 lens 488/4536 e 0 to 0 dl 1788572560 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572549.980086:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ca8c8700 with x1875453360599808 ext(0->4095) 00002000:00010000:2.0:1788572549.980090:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ca8c8700 00000100:00100000:3.0:1788572549.980404:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572549.981820:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360599936 00000100:00100000:2.0:1788572549.981834:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360599936 00000100:00100000:2.0:1788572549.981950:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 352 00000100:00100000:2.0:1788572549.981961:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:2347:x1875453360599936:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572549.981981:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572549.982058:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb165182 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x83:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0e91 expref: 287 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572549.982086:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x83:0x0].0x0 (ffff9627c613b000) 00010000:00010000:2.0:1788572549.982092:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572549.982168:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+286:2347:x1875453360599936:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 209us (1764us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572549.982175:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 352 00000100:00100000:3.0:1788572550.010229:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.010312:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360600192 00000100:00100000:1.0:1788572550.010325:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360600192 00000100:00100000:1.0:1788572550.010388:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1024 00000100:00100000:1.0:1788572550.010399:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+286:35867:x1875453360600192:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572550.010819:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:35867:x1875453360600192:12345-192.168.202.40@tcp:35:diff.0 Request processed in 420us (597us total) trans 4294970910 rc 0/0 00000100:00100000:1.0:1788572550.010827:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1024 00000100:00100000:1.0:1788572550.021228:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.021345:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360600320 00000100:00100000:2.0:1788572550.021356:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360600320 00000100:00100000:2.0:1788572550.021422:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4790 00000100:00100000:2.0:1788572550.021443:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:35867:x1875453360600320:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.021470:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.021494:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1651a5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0f7f expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.021570:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb1651ac lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.021585:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb1651ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.021598:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb1651ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.021635:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00000004:00200000:2.0:1788572550.021639:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.021658:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb1651ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.021684:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1651a5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f7f expref: 289 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.021703:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715800/0x85ace67feb1651ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f7f expref: 288 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.021712:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.021715:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572550.021720:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715800, rc 0) 00000100:00100000:2.0:1788572550.021784:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+288:35867:x1875453360600320:12345-192.168.202.40@tcp:101:diff.0 Request processed in 343us (566us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.021791:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4790 00000100:00100000:3.0:1788572550.025943:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.026263:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360600576 00000100:00100000:1.0:1788572550.026275:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360600576 00000100:00100000:1.0:1788572550.026340:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4791 00000100:00100000:1.0:1788572550.026352:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+288:35867:x1875453360600576:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572550.026374:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.026397:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1651b3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0f86 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572550.026515:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 16384 00010000:00010000:1.0:1788572550.026642:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1651b3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f86 expref: 290 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.026656:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:1.0:1788572550.026659:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572550.026664:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb1651b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f86 expref: 290 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.026684:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c246c000, rc 0) 00000100:00100000:1.0:1788572550.026752:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+289:35867:x1875453360600576:12345-192.168.202.40@tcp:101:diff.0 Request processed in 402us (817us total) trans 4294970911 rc 301/301 00000100:00100000:1.0:1788572550.026760:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4791 00000100:00100000:3.0:1788572550.031116:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.031232:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360600704 00000100:00100000:0.0:1788572550.031243:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360600704 00000100:00100000:0.0:1788572550.031300:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1025 00000100:00100000:0.0:1788572550.031310:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+289:35867:x1875453360600704:12345-192.168.202.40@tcp:37:diff.0 00000100:00100000:0.0:1788572550.035853:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+289:35867:x1875453360600704:12345-192.168.202.40@tcp:37:diff.0 Request processed in 4532us (4735us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.035867:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1025 00000100:00100000:3.0:1788572550.042467:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.042561:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360600960 00000100:00100000:0.0:1788572550.042572:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360600960 00000100:00100000:0.0:1788572550.042628:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1026 00000100:00100000:0.0:1788572550.042639:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+289:35867:x1875453360600960:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.042882:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+290:35867:x1875453360600960:12345-192.168.202.40@tcp:35:diff.0 Request processed in 244us (425us total) trans 4294970912 rc 0/0 00000100:00100000:0.0:1788572550.042892:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1026 00000100:00100000:3.0:1788572550.048140:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.048252:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360601088 00000100:00100000:0.0:1788572550.048264:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360601088 00000100:00100000:0.0:1788572550.048327:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4792 00000100:00100000:0.0:1788572550.048338:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+290:35867:x1875453360601088:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.048363:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.048385:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1651c1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0f8d expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048488:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048504:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048516:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048534:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048541:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048548:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048830:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb1651d6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x80:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048844:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb1651d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048852:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb1651d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.048885:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x80:0x0]: returning size 3 00000004:00200000:0.0:1788572550.048890:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572550.048909:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb1651d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048922:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb1651d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048930:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048937:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048944:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048951:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea2a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.048956:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048964:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048977:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:0.0:1788572550.048981:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.048986:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.048992:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb1651c8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049109:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049117:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049123:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049129:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5b05600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.049131:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049138:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049147:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x9beb5ca3 (ffff9627d0ee2300) 00010000:00010000:0.0:1788572550.049149:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.049152:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049159:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5b05600/0x85ace67feb1651cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049192:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb1651c1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f8d expref: 292 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049207:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93e00/0x85ace67feb1651d6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f8d expref: 291 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.049214:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d1866e00) 00010000:00010000:0.0:1788572550.049216:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.049218:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93e00, rc 0) 00000100:00100000:0.0:1788572550.049396:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+291:35867:x1875453360601088:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1058us (1265us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.049403:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4792 00000100:00100000:3.0:1788572550.058539:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.062100:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360601216 00000100:00100000:1.0:1788572550.062118:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360601216 00000100:00100000:0.0:1788572550.064401:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1380 00000100:00100000:0.0:1788572550.064416:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2347:x1875453360601216:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.064444:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.064469:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c0a14a2/0x85ace67feb1651dd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0f94 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.064499:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3d:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:0.0:1788572550.064520:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.064524:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.064536:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c0a14a2/0x85ace67feb1651dd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f94 expref: 74 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.064557:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002c0a14a2/0x85ace67feb1651dd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f94 expref: 74 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.064577:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92800, rc 0) 00000100:00100000:0.0:1788572550.064654:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2347:x1875453360601216:12345-192.168.202.40@tcp:101:diff.0 Request processed in 240us (6125us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.064661:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1380 00000100:00100000:2.0:1788572550.068916:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.069058:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360601472 00000100:00100000:3.0:1788572550.069068:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360601472 00000100:00100000:3.0:1788572550.069135:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4793 00000100:00100000:3.0:1788572550.069150:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+291:35867:x1875453360601472:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.069300:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.069324:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1651e4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0f9b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.069443:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1651eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x80:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.069462:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1651eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.069474:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1651eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.069491:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x80:0x0]: returning size 3 00010000:00010000:3.0:1788572550.069631:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1651e4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f9b expref: 294 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.069650:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1651eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f9b expref: 293 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.069661:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d1866e00) 00010000:00010000:3.0:1788572550.069663:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572550.069667:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a91c00, rc 0) 00000100:00100000:3.0:1788572550.069771:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:35867:x1875453360601472:12345-192.168.202.40@tcp:101:diff.0 Request processed in 623us (873us total) trans 4294970913 rc 0/0 00000100:00100000:3.0:1788572550.069779:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4793 00000100:00100000:1.0:1788572550.082115:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.082212:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360601600 00000100:00100000:2.0:1788572550.082223:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360601600 00002000:00100000:2.0:1788572550.082270:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x3d:0x0] (0->4095) req@ffff9626c4bb8e00 x1875453360601600/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.082282:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bb8e00 with x1875453360601600 ext(0->4095) 00002000:00010000:2.0:1788572550.082288:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bb8e00 00000100:00100000:2.0:1788572550.082321:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 335 00000100:00100000:2.0:1788572550.082331:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360601600:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.082572:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360601600:12345-192.168.202.40@tcp:3:diff.0 Request processed in 242us (467us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.082580:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 335 00002000:00100000:2.0:1788572550.082587:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x3d:0x0] (0->4095) req@ffff9626c4bb8e00 x1875453360601600/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.082596:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bb8e00 with x1875453360601600 ext(0->4095) 00002000:00010000:2.0:1788572550.082599:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bb8e00 00000100:00100000:0.0:1788572550.085147:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.085352:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360601728 00000100:00100000:3.0:1788572550.085362:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360601728 00000100:00100000:3.0:1788572550.085414:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 353 00000100:00100000:3.0:1788572550.085436:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:2350:x1875453360601728:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.085454:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.085481:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1651eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x80:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0f9b expref: 293 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.085500:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x80:0x0].0x0 (ffff9627d1866e00) 00010000:00010000:3.0:1788572550.085506:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.085550:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+292:2350:x1875453360601728:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 128us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.085557:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 353 00000100:00100000:3.0:1788572550.089277:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.089376:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360601984 00000100:00100000:0.0:1788572550.089387:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360601984 00000100:00100000:0.0:1788572550.089499:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1027 00000100:00100000:0.0:1788572550.089511:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+292:35867:x1875453360601984:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.089928:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:35867:x1875453360601984:12345-192.168.202.40@tcp:35:diff.0 Request processed in 419us (661us total) trans 4294970914 rc 0/0 00000100:00100000:0.0:1788572550.089937:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1027 00000100:00100000:1.0:1788572550.092750:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.093124:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360602112 00000100:00100000:2.0:1788572550.093135:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360602112 00000100:00100000:2.0:1788572550.093340:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4794 00000100:00100000:2.0:1788572550.093352:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:35867:x1875453360602112:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.093377:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.093400:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1651f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fa2 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093486:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093500:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093511:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093525:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093532:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093537:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093827:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16520e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x77:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093837:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16520e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093845:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16520e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.093879:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x77:0x0]: returning size 3 00000004:00200000:2.0:1788572550.093883:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.093918:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16520e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093931:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16520e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093938:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093945:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093951:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093957:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3714000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.093961:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093969:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093980:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.093984:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.093988:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.093995:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165200 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094141:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094148:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094155:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094160:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.094163:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094169:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094253:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x6e9b5ca3 (ffff9627c613b800) 00010000:00010000:2.0:1788572550.094257:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.094260:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094266:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165207 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094300:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1651f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fa2 expref: 295 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094314:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96280094d600/0x85ace67feb16520e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fa2 expref: 294 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.094321:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627c613b100) 00010000:00010000:2.0:1788572550.094324:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.094326:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96280094d600, rc 0) 00000100:00100000:2.0:1788572550.094444:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+294:35867:x1875453360602112:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1094us (1702us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.094455:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4794 00000100:00100000:3.0:1788572550.097641:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.097718:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360602240 00000100:00100000:2.0:1788572550.097729:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360602240 00000100:00100000:2.0:1788572550.097796:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1381 00000100:00100000:2.0:1788572550.097807:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2348:x1875453360602240:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.097826:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.097859:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb165215 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fa9 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572550.097891:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3c:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:2.0:1788572550.097921:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3c:0x0].0x0 since=898s v=(0 0) 00010000:00010000:2.0:1788572550.097926:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572550.097937:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb165215 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fa9 expref: 75 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.097956:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c3bdfee4/0x85ace67feb165215 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fa9 expref: 75 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.097968:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c2bade00, rc 0) 00000100:00100000:2.0:1788572550.098064:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2348:x1875453360602240:12345-192.168.202.40@tcp:101:diff.0 Request processed in 258us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.098070:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1381 00000100:00100000:3.0:1788572550.102484:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.102639:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360602496 00000100:00100000:2.0:1788572550.102651:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360602496 00000100:00100000:2.0:1788572550.102722:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4795 00000100:00100000:2.0:1788572550.102732:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+294:35867:x1875453360602496:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.102757:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.102787:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16521c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fb0 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.102933:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165223 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x77:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.102947:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165223 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.102959:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165223 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.102969:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x77:0x0]: returning size 3 00010000:00010000:2.0:1788572550.103271:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb16521c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fb0 expref: 297 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.103287:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165223 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fb0 expref: 296 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.103293:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627c613b100) 00010000:00010000:2.0:1788572550.103295:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.103298:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715200, rc 0) 00000100:00100000:2.0:1788572550.103359:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+296:35867:x1875453360602496:12345-192.168.202.40@tcp:101:diff.0 Request processed in 627us (890us total) trans 4294970915 rc 0/0 00000100:00100000:2.0:1788572550.103364:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4795 00000100:00100000:1.0:1788572550.108944:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.108994:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.109173:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360602624 00000100:00100000:2.0:1788572550.109194:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360602624 00002000:00100000:2.0:1788572550.109246:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x3c:0x0] (0->4095) req@ffff9626c4bbad80 x1875453360602624/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.109259:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bbad80 with x1875453360602624 ext(0->4095) 00002000:00010000:2.0:1788572550.109265:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bbad80 00000100:00100000:2.0:1788572550.109289:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 336 00000100:00100000:2.0:1788572550.109298:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2349:x1875453360602624:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.109556:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2349:x1875453360602624:12345-192.168.202.40@tcp:3:diff.0 Request processed in 258us (622us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.109565:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 336 00002000:00100000:2.0:1788572550.109573:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x3c:0x0] (0->4095) req@ffff9626c4bbad80 x1875453360602624/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.109582:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4bbad80 with x1875453360602624 ext(0->4095) 00002000:00010000:2.0:1788572550.109585:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4bbad80 00000100:00100000:3.0:1788572550.109924:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360602752 00000100:00100000:3.0:1788572550.109929:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360602752 00000100:00100000:3.0:1788572550.109977:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 354 00000100:00100000:3.0:1788572550.109987:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+296:2347:x1875453360602752:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.110000:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.110055:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165223 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x77:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fb0 expref: 296 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.110071:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x77:0x0].0x0 (ffff9627c613b100) 00010000:00010000:3.0:1788572550.110076:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.110111:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+295:2347:x1875453360602752:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 126us (1116us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.110117:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 354 00000100:00100000:1.0:1788572550.115273:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.115422:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360603008 00000100:00100000:2.0:1788572550.115441:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360603008 00000100:00100000:2.0:1788572550.115505:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1028 00000100:00100000:2.0:1788572550.115516:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+295:35867:x1875453360603008:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572550.115973:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+296:35867:x1875453360603008:12345-192.168.202.40@tcp:35:diff.0 Request processed in 458us (709us total) trans 4294970916 rc 0/0 00000100:00100000:2.0:1788572550.115981:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1028 00000100:00100000:1.0:1788572550.120457:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.120546:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360603136 00000100:00100000:0.0:1788572550.120556:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360603136 00000100:00100000:0.0:1788572550.120618:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4796 00000100:00100000:0.0:1788572550.120629:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+296:35867:x1875453360603136:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.120652:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.120675:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb165231 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fb7 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.120748:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.120763:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.120779:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.120794:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.120801:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.120808:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121182:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb165246 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121192:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb165246 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121199:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb165246 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.121235:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7f:0x0]: returning size 3 00000004:00200000:0.0:1788572550.121240:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572550.121260:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb165246 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121272:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb165246 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121280:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121289:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121296:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121303:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.121307:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121315:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121327:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:0.0:1788572550.121330:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.121335:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121341:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93c00/0x85ace67feb165238 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121353:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121358:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121363:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121368:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627f5a93a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.121371:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121376:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121383:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0xe44515a3 (ffff9627c29dad00) 00010000:00010000:0.0:1788572550.121385:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.121387:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121392:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a93a00/0x85ace67feb16523f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121422:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92200/0x85ace67feb165231 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fb7 expref: 298 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121629:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627f5a92600/0x85ace67feb165246 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fb7 expref: 297 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.121639:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7f:0x0].0x0 (ffff9627fb58d700) 00010000:00010000:0.0:1788572550.121641:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.121644:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92600, rc 0) 00000100:00100000:0.0:1788572550.121703:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+297:35867:x1875453360603136:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1075us (1255us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.121710:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4796 00000100:00100000:0.0:1788572550.126577:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.126666:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360603264 00000100:00100000:3.0:1788572550.126678:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360603264 00000100:00100000:3.0:1788572550.126748:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1382 00000100:00100000:3.0:1788572550.126761:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2349:x1875453360603264:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.126784:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.126809:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000254843a3/0x85ace67feb16524d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fbe expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572550.126843:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3e:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:3.0:1788572550.126881:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3e:0x0].0x0 since=898s v=(0 0) 00010000:00010000:3.0:1788572550.126885:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572550.126895:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000254843a3/0x85ace67feb16524d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fbe expref: 76 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.126920:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000254843a3/0x85ace67feb16524d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fbe expref: 76 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.126940:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a91400, rc 0) 00000100:00100000:3.0:1788572550.127037:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2349:x1875453360603264:12345-192.168.202.40@tcp:101:diff.0 Request processed in 279us (469us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.127045:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1382 00000100:00100000:1.0:1788572550.131935:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.132087:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360603520 00000100:00100000:2.0:1788572550.132098:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360603520 00000100:00100000:2.0:1788572550.132158:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4797 00000100:00100000:2.0:1788572550.132168:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+297:35867:x1875453360603520:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.132198:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.132221:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165254 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fc5 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.132327:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16525b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.132340:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16525b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.132351:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16525b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.132362:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7f:0x0]: returning size 3 00010000:00010000:2.0:1788572550.132510:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165254 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fc5 expref: 300 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.132529:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16525b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fc5 expref: 299 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.132538:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7f:0x0].0x0 (ffff9627fb58d700) 00010000:00010000:2.0:1788572550.132540:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.132544:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715200, rc 0) 00000100:00100000:2.0:1788572550.132608:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+299:35867:x1875453360603520:12345-192.168.202.40@tcp:101:diff.0 Request processed in 441us (685us total) trans 4294970917 rc 0/0 00000100:00100000:2.0:1788572550.132616:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4797 00000100:00100000:2.0:1788572550.141120:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.141170:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.141314:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360603648 00000100:00100000:1.0:1788572550.141326:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360603648 00002000:00100000:1.0:1788572550.141382:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x3e:0x0] (0->4095) req@ffff9627e7876d80 x1875453360603648/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.141394:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7876d80 with x1875453360603648 ext(0->4095) 00002000:00010000:1.0:1788572550.141400:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7876d80 00000100:00100000:1.0:1788572550.141448:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 337 00000100:00100000:1.0:1788572550.141467:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2350:x1875453360603648:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.141740:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2350:x1875453360603648:12345-192.168.202.40@tcp:3:diff.0 Request processed in 274us (631us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.141748:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 337 00002000:00100000:1.0:1788572550.141755:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x3e:0x0] (0->4095) req@ffff9627e7876d80 x1875453360603648/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.141763:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7876d80 with x1875453360603648 ext(0->4095) 00002000:00010000:1.0:1788572550.141767:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7876d80 00000100:00100000:3.0:1788572550.142328:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360603776 00000100:00100000:3.0:1788572550.142334:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360603776 00000100:00100000:3.0:1788572550.142386:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 355 00000100:00100000:3.0:1788572550.142394:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+299:2350:x1875453360603776:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.142406:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.142441:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16525b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fc5 expref: 299 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.142468:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7f:0x0].0x0 (ffff9627fb58d700) 00010000:00010000:3.0:1788572550.142473:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.142512:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+298:2350:x1875453360603776:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 120us (1341us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.142520:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 355 00000100:00100000:2.0:1788572550.149143:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.149308:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360604032 00000100:00100000:0.0:1788572550.149319:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360604032 00000100:00100000:0.0:1788572550.149385:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1029 00000100:00100000:0.0:1788572550.149396:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+298:35867:x1875453360604032:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.149885:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+299:35867:x1875453360604032:12345-192.168.202.40@tcp:35:diff.0 Request processed in 488us (755us total) trans 4294970918 rc 0/0 00000100:00100000:0.0:1788572550.149893:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1029 00000100:00100000:0.0:1788572550.158666:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.158798:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360604160 00000100:00100000:2.0:1788572550.158809:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360604160 00000100:00100000:2.0:1788572550.159075:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4798 00000100:00100000:2.0:1788572550.159087:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+299:35867:x1875453360604160:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.159115:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.159138:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165269 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fcc expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159223:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159238:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159250:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159265:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159271:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159277:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159577:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8600/0x85ace67feb16527e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159586:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8600/0x85ace67feb16527e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159594:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8600/0x85ace67feb16527e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.159630:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7e:0x0]: returning size 3 00000004:00200000:2.0:1788572550.159634:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.159652:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8600/0x85ace67feb16527e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159663:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8600/0x85ace67feb16527e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159671:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159676:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159683:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159688:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.159692:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159700:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159711:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.159714:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.159718:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159725:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165270 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159739:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159744:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159749:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159753:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.159755:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159760:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159766:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x2c9ecea3 (ffff9627c613b800) 00010000:00010000:2.0:1788572550.159767:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.159769:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159775:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165277 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159802:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165269 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fcc expref: 301 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159814:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8600/0x85ace67feb16527e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fcc expref: 300 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.159826:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7e:0x0].0x0 (ffff9627c613bd00) 00010000:00010000:2.0:1788572550.159828:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.159830:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8600, rc 0) 00000100:00100000:2.0:1788572550.159915:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+300:35867:x1875453360604160:12345-192.168.202.40@tcp:101:diff.0 Request processed in 830us (1260us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.159922:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4798 00000100:00100000:1.0:1788572550.166119:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.167677:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360604288 00000100:00100000:0.0:1788572550.167691:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360604288 00000100:00100000:0.0:1788572550.167767:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1383 00000100:00100000:0.0:1788572550.167777:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+60:2349:x1875453360604288:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.167795:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.167819:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006cbb7bae/0x85ace67feb165285 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fd3 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.167848:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x41:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:0.0:1788572550.167873:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.167877:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.167887:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006cbb7bae/0x85ace67feb165285 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fd3 expref: 61 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.167919:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006cbb7bae/0x85ace67feb165285 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fd3 expref: 61 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.167932:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93c00, rc 0) 00000100:00100000:0.0:1788572550.167995:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+61:2349:x1875453360604288:12345-192.168.202.40@tcp:101:diff.0 Request processed in 220us (1887us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.168035:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1383 00000100:00100000:1.0:1788572550.171934:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.172052:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360604544 00000100:00100000:3.0:1788572550.172063:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360604544 00000100:00100000:3.0:1788572550.172121:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4799 00000100:00100000:3.0:1788572550.172132:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+300:35867:x1875453360604544:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.172152:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.172173:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb16528c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fda expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.172312:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb165293 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.172325:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb165293 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.172336:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb165293 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.172347:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7e:0x0]: returning size 3 00010000:00010000:3.0:1788572550.172500:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91e00/0x85ace67feb16528c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fda expref: 303 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.172519:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb165293 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fda expref: 302 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.172529:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7e:0x0].0x0 (ffff9627c613bd00) 00010000:00010000:3.0:1788572550.172532:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572550.172537:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a91c00, rc 0) 00000100:00100000:3.0:1788572550.172599:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+302:35867:x1875453360604544:12345-192.168.202.40@tcp:101:diff.0 Request processed in 469us (674us total) trans 4294970919 rc 0/0 00000100:00100000:3.0:1788572550.172607:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4799 00000100:00100000:0.0:1788572550.177992:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.178196:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360604672 00000100:00100000:1.0:1788572550.178206:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360604672 00002000:00100000:1.0:1788572550.178259:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x41:0x0] (0->4095) req@ffff9627e786df80 x1875453360604672/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.178270:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786df80 with x1875453360604672 ext(0->4095) 00002000:00010000:1.0:1788572550.178275:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786df80 00000100:00100000:1.0:1788572550.178301:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 338 00000100:00100000:1.0:1788572550.178310:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+61:2349:x1875453360604672:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.178584:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+61:2349:x1875453360604672:12345-192.168.202.40@tcp:3:diff.0 Request processed in 274us (603us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.178592:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 338 00002000:00100000:1.0:1788572550.178599:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x41:0x0] (0->4095) req@ffff9627e786df80 x1875453360604672/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.178607:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786df80 with x1875453360604672 ext(0->4095) 00002000:00010000:1.0:1788572550.178610:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786df80 00000100:00100000:1.0:1788572550.182945:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.182994:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.183133:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360604928 00000100:00100000:0.0:1788572550.183145:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360604928 00000100:00100000:0.0:1788572550.183222:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1030 00000100:00100000:0.0:1788572550.183234:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+302:35867:x1875453360604928:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.183666:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+303:35867:x1875453360604928:12345-192.168.202.40@tcp:35:diff.0 Request processed in 432us (731us total) trans 4294970920 rc 0/0 00000100:00100000:0.0:1788572550.183674:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1030 00000100:00100000:2.0:1788572550.186373:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360605056 00000100:00100000:2.0:1788572550.186389:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360605056 00000100:00100000:0.0:1788572550.186539:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 356 00000100:00100000:0.0:1788572550.186551:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+303:2348:x1875453360605056:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572550.186570:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572550.186594:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb165293 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fda expref: 303 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.186615:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7e:0x0].0x0 (ffff9627c613bd00) 00010000:00010000:0.0:1788572550.186620:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572550.186677:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+302:2348:x1875453360605056:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 129us (3682us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.186682:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 356 00000100:00100000:0.0:1788572550.189293:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.189392:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360605184 00000100:00100000:2.0:1788572550.189402:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360605184 00000100:00100000:2.0:1788572550.189470:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4800 00000100:00100000:2.0:1788572550.189481:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+302:35867:x1875453360605184:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.189506:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.189526:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fe1 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189618:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189634:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189645:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189661:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189673:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189679:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189985:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8400/0x85ace67feb1652b6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.189995:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8400/0x85ace67feb1652b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190038:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8400/0x85ace67feb1652b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.190079:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7d:0x0]: returning size 3 00000004:00200000:2.0:1788572550.190083:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.190104:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8400/0x85ace67feb1652b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190115:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8400/0x85ace67feb1652b6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190123:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190131:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190137:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190147:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.190151:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190158:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190171:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.190174:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.190595:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190604:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190617:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190624:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190630:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190634:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.190637:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190643:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190655:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x74f887a3 (ffff9627c613b800) 00010000:00010000:2.0:1788572550.190658:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.190660:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190666:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652af lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190700:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fe1 expref: 304 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190712:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8400/0x85ace67feb1652b6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fe1 expref: 303 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.190718:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627c613ba00) 00010000:00010000:2.0:1788572550.190720:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.190722:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8400, rc 0) 00000100:00100000:2.0:1788572550.190825:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+303:35867:x1875453360605184:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1345us (1647us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.190833:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4800 00000100:00100000:0.0:1788572550.194862:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.195214:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360605312 00000100:00100000:2.0:1788572550.195224:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360605312 00000100:00100000:2.0:1788572550.195290:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1384 00000100:00100000:2.0:1788572550.195300:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2349:x1875453360605312:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.195317:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.195337:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004345f1f7/0x85ace67feb1652bd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fe8 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572550.195370:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x41:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:2.0:1788572550.195389:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=898s v=(0 0) 00010000:00010000:2.0:1788572550.195393:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x41:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572550.195402:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004345f1f7/0x85ace67feb1652bd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fe8 expref: 77 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.195419:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004345f1f7/0x85ace67feb1652bd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x41:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fe8 expref: 77 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.195880:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9800, rc 0) 00000100:00100000:2.0:1788572550.196256:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+77:2349:x1875453360605312:12345-192.168.202.40@tcp:101:diff.0 Request processed in 955us (1414us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.196264:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1384 00000100:00100000:3.0:1788572550.201137:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.201314:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360605568 00000100:00100000:2.0:1788572550.201324:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360605568 00000100:00100000:2.0:1788572550.201378:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4801 00000100:00100000:2.0:1788572550.201388:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+303:35867:x1875453360605568:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.201409:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.201440:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0fef expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.201555:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.201570:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.201583:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.201593:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7d:0x0]: returning size 3 00010000:00010000:2.0:1788572550.201723:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fef expref: 306 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.201740:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fef expref: 305 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.201749:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627c613ba00) 00010000:00010000:2.0:1788572550.201753:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.201756:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9a00, rc 0) 00000100:00100000:2.0:1788572550.201826:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:35867:x1875453360605568:12345-192.168.202.40@tcp:101:diff.0 Request processed in 439us (699us total) trans 4294970921 rc 0/0 00000100:00100000:2.0:1788572550.201833:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4801 00000100:00100000:0.0:1788572550.209157:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.209203:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.209282:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360605824 00000100:00100000:2.0:1788572550.209295:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360605824 00000100:00100000:2.0:1788572550.209365:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 357 00000100:00100000:2.0:1788572550.209375:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:2349:x1875453360605824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572550.209391:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572550.209415:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0fef expref: 305 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.209445:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7d:0x0].0x0 (ffff9627c613ba00) 00010000:00010000:2.0:1788572550.209450:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572550.209516:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+304:2349:x1875453360605824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 142us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.209524:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 357 00000100:00100000:1.0:1788572550.210350:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360605696 00000100:00100000:1.0:1788572550.210359:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360605696 00002000:00100000:1.0:1788572550.210399:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x41:0x0] (0->4095) req@ffff9627e786ca80 x1875453360605696/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.210410:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786ca80 with x1875453360605696 ext(0->4095) 00002000:00010000:1.0:1788572550.210416:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e786ca80 00000100:00100000:1.0:1788572550.210459:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 339 00000100:00100000:1.0:1788572550.210468:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+77:2350:x1875453360605696:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.211220:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+77:2350:x1875453360605696:12345-192.168.202.40@tcp:3:diff.0 Request processed in 751us (2069us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.211229:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 339 00002000:00100000:1.0:1788572550.211237:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x41:0x0] (0->4095) req@ffff9627e786ca80 x1875453360605696/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.211245:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786ca80 with x1875453360605696 ext(0->4095) 00002000:00010000:1.0:1788572550.211248:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e786ca80 00000100:00100000:0.0:1788572550.221721:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.221818:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360606080 00000100:00100000:1.0:1788572550.221830:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360606080 00000100:00100000:1.0:1788572550.221890:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1031 00000100:00100000:1.0:1788572550.221911:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+304:35867:x1875453360606080:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572550.222543:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:35867:x1875453360606080:12345-192.168.202.40@tcp:35:diff.0 Request processed in 632us (832us total) trans 4294970922 rc 0/0 00000100:00100000:1.0:1788572550.222554:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1031 00000100:00100000:3.0:1788572550.230523:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.230631:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360606208 00000100:00100000:2.0:1788572550.230641:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360606208 00000100:00100000:2.0:1788572550.230700:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4802 00000100:00100000:2.0:1788572550.230710:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:35867:x1875453360606208:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.230736:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.230757:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652d9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ff6 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.230820:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.230835:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.230857:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.230873:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.230881:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.230887:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231198:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9400/0x85ace67feb1652ee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231212:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9400/0x85ace67feb1652ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231218:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9400/0x85ace67feb1652ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.231251:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7c:0x0]: returning size 3 00000004:00200000:2.0:1788572550.231255:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.231272:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9400/0x85ace67feb1652ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231284:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9400/0x85ace67feb1652ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231293:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231299:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231307:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231313:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.231318:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231326:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231338:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.231342:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.231348:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231355:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb1652e0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231366:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231372:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231378:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231384:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.231387:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231393:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231399:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0xbd5240a3 (ffff9627c613b800) 00010000:00010000:2.0:1788572550.231402:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.231405:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231410:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1652e7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231452:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652d9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ff6 expref: 307 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231466:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9400/0x85ace67feb1652ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ff6 expref: 306 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.231473:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7c:0x0].0x0 (ffff9627c613b500) 00010000:00010000:2.0:1788572550.231475:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.231477:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9400, rc 0) 00000100:00100000:2.0:1788572550.231536:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+306:35867:x1875453360606208:12345-192.168.202.40@tcp:101:diff.0 Request processed in 828us (1023us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.231544:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4802 00000100:00100000:0.0:1788572550.235945:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.236255:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360606336 00000100:00100000:3.0:1788572550.236265:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360606336 00000100:00100000:3.0:1788572550.236324:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1385 00000100:00100000:3.0:1788572550.236334:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+61:2350:x1875453360606336:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.236353:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.236374:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090235507/0x85ace67feb1652f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f0ffd expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572550.236402:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x3e:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:3.0:1788572550.236422:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=898s v=(0 0) 00010000:00010000:3.0:1788572550.236438:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572550.236449:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090235507/0x85ace67feb1652f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ffd expref: 62 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.236469:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090235507/0x85ace67feb1652f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f0ffd expref: 62 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.236482:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a91e00, rc 0) 00000100:00100000:3.0:1788572550.236536:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360606336:12345-192.168.202.40@tcp:101:diff.0 Request processed in 203us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.236544:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1385 00000100:00100000:1.0:1788572550.241505:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.241596:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360606592 00000100:00100000:2.0:1788572550.241607:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360606592 00000100:00100000:2.0:1788572550.241669:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4803 00000100:00100000:2.0:1788572550.241679:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+306:35867:x1875453360606592:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.241701:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.241722:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652fc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1004 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.241827:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165303 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.241841:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165303 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.241853:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165303 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.241863:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7c:0x0]: returning size 3 00010000:00010000:2.0:1788572550.242037:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb1652fc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1004 expref: 309 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.242056:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165303 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1004 expref: 308 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.242065:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7c:0x0].0x0 (ffff9627c613b500) 00010000:00010000:2.0:1788572550.242068:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.242072:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9000, rc 0) 00000100:00100000:2.0:1788572550.242129:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+308:35867:x1875453360606592:12345-192.168.202.40@tcp:101:diff.0 Request processed in 450us (634us total) trans 4294970923 rc 0/0 00000100:00100000:2.0:1788572550.242137:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4803 00000100:00100000:0.0:1788572550.246852:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.246986:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360606720 00000100:00100000:1.0:1788572550.246997:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360606720 00000100:00100000:1.0:1788572550.247096:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 358 00000100:00100000:1.0:1788572550.247107:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2eff450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+308:2347:x1875453360606720:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572550.247127:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572550.247153:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165303 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1004 expref: 308 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.247173:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7c:0x0].0x0 (ffff9627c613b500) 00010000:00010000:1.0:1788572550.247191:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572550.247247:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2eff450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+307:2347:x1875453360606720:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 142us (406us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.247255:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 358 00000100:00100000:0.0:1788572550.251085:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.251227:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360606848 00000100:00100000:2.0:1788572550.251238:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360606848 00002000:00100000:2.0:1788572550.251299:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x3e:0x0] (0->4095) req@ffff9627fb30ca80 x1875453360606848/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.251311:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30ca80 with x1875453360606848 ext(0->4095) 00002000:00010000:2.0:1788572550.251316:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30ca80 00000100:00100000:2.0:1788572550.251346:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 340 00000100:00100000:2.0:1788572550.251355:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360606848:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.251651:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360606848:12345-192.168.202.40@tcp:3:diff.0 Request processed in 295us (576us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.251660:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 340 00002000:00100000:2.0:1788572550.251667:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x3e:0x0] (0->4095) req@ffff9627fb30ca80 x1875453360606848/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.251675:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30ca80 with x1875453360606848 ext(0->4095) 00002000:00010000:2.0:1788572550.251679:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30ca80 00000100:00100000:2.0:1788572550.257338:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.257550:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360607104 00000100:00100000:3.0:1788572550.257560:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360607104 00000100:00100000:3.0:1788572550.257619:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1032 00000100:00100000:3.0:1788572550.257629:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+307:35867:x1875453360607104:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572550.258080:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+308:35867:x1875453360607104:12345-192.168.202.40@tcp:35:diff.0 Request processed in 451us (750us total) trans 4294970924 rc 0/0 00000100:00100000:3.0:1788572550.258089:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1032 00000100:00100000:0.0:1788572550.261783:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.261910:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360607232 00000100:00100000:2.0:1788572550.261921:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360607232 00000100:00100000:2.0:1788572550.261985:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4804 00000100:00100000:2.0:1788572550.261996:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+308:35867:x1875453360607232:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.262060:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.262084:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165311 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f100b expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262157:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262171:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262183:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262198:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262206:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262212:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262525:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8000/0x85ace67feb165326 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262540:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8000/0x85ace67feb165326 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262547:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8000/0x85ace67feb165326 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.262578:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7b:0x0]: returning size 3 00000004:00200000:2.0:1788572550.262582:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.262598:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8000/0x85ace67feb165326 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262609:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8000/0x85ace67feb165326 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262616:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262622:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262628:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262633:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.262637:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262644:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262654:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.262657:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.262661:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262667:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165318 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262679:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262684:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262690:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262693:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.262696:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262701:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262707:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x5abf9a3 (ffff9627c613b800) 00010000:00010000:2.0:1788572550.262709:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.262712:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262717:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16531f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262747:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165311 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f100b expref: 310 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262760:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8000/0x85ace67feb165326 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f100b expref: 309 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.262767:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7b:0x0].0x0 (ffff9627c613b700) 00010000:00010000:2.0:1788572550.262768:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.262771:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8000, rc 0) 00000100:00100000:2.0:1788572550.262832:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+309:35867:x1875453360607232:12345-192.168.202.40@tcp:101:diff.0 Request processed in 839us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.262839:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4804 00010000:00010000:2.0:1788572550.266334:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572550.268995:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.269125:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360607360 00000100:00100000:0.0:1788572550.269135:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360607360 00000100:00100000:0.0:1788572550.269205:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1386 00000100:00100000:0.0:1788572550.269215:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+62:2350:x1875453360607360:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.269235:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.269256:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003f3e78e1/0x85ace67feb16532d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1012 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.269286:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x3d:0x0] initial LVB size: 3, mtime: 0x6a9b7368, atime: 0x6a9b7368, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:0.0:1788572550.269307:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3d:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.269313:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.269323:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003f3e78e1/0x85ace67feb16532d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1012 expref: 63 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.269341:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003f3e78e1/0x85ace67feb16532d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1012 expref: 63 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.269355:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93600, rc 0) 00000100:00100000:0.0:1788572550.269413:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+63:2350:x1875453360607360:12345-192.168.202.40@tcp:101:diff.0 Request processed in 199us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.269419:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1386 00000100:00100000:1.0:1788572550.272695:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.272923:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360607488 00000100:00100000:2.0:1788572550.272933:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360607488 00000100:00100000:2.0:1788572550.272995:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 378 00000100:00100000:2.0:1788572550.273032:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+309:2347:x1875453360607488:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572550.273068:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.273088:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165334 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1019 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273143:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273156:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273167:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273193:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273199:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273205:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273510:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9200/0x85ace67feb165349 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273520:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9200/0x85ace67feb165349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273527:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9200/0x85ace67feb165349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.273545:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7a:0x0]: returning size 3 00000004:00200000:2.0:1788572550.273549:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.273568:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9200/0x85ace67feb165349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273580:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9200/0x85ace67feb165349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273588:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273594:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273605:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273611:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.273615:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273622:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273635:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.273638:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.273643:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273649:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb16533b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273661:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273667:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273673:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273677:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.273680:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273690:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273697:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x4e05b2a3 (ffff9627c613b800) 00010000:00010000:2.0:1788572550.273700:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.273703:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273708:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165342 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273735:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165334 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1019 expref: 311 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273747:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9200/0x85ace67feb165349 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1019 expref: 310 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273754:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627c613bf00) 00010000:00010000:2.0:1788572550.273756:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.273759:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9200, rc 0) 00010000:00010000:2.0:1788572550.273764:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.273771:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165350 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1020 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273795:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273803:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273815:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273827:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273833:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.273839:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274083:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8e00/0x85ace67feb165365 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x79:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274092:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8e00/0x85ace67feb165365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274099:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8e00/0x85ace67feb165365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.274117:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x79:0x0]: returning size 3 00000004:00200000:2.0:1788572550.274120:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.274131:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8e00/0x85ace67feb165365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274139:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8e00/0x85ace67feb165365 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274145:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274154:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274161:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274166:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.274169:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274174:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274183:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.274186:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.274189:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274193:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165357 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274202:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274207:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274212:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274217:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.274219:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274225:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274231:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x965f6ba3 (ffff9627d20cb300) 00010000:00010000:2.0:1788572550.274233:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.274236:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274241:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb16535e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274263:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165350 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1020 expref: 312 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274273:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8e00/0x85ace67feb165365 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1020 expref: 311 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274280:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x79:0x0].0x0 (ffff9627ffc19300) 00010000:00010000:2.0:1788572550.274282:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.274284:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8e00, rc 0) 00010000:00010000:2.0:1788572550.274288:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.274295:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16536c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1027 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274317:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274325:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274333:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274344:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274350:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274357:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274533:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165381 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x78:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274542:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165381 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274548:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165381 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.274561:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x78:0x0]: returning size 3 00000004:00200000:2.0:1788572550.274563:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.274573:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165381 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274581:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165381 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274588:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274597:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274604:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274608:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.274611:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274617:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274624:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.274626:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.274629:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274634:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165373 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274643:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274648:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274653:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274657:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.274660:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274666:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274671:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0xdeb924a3 (ffff9627ffc19400) 00010000:00010000:2.0:1788572550.274674:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.274676:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274681:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb16537a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274700:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16536c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1027 expref: 313 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274710:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3714000/0x85ace67feb165381 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1027 expref: 312 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274716:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x78:0x0].0x0 (ffff9627ffc19600) 00010000:00010000:2.0:1788572550.274718:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.274720:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3714000, rc 0) 00010000:00010000:2.0:1788572550.274724:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.274731:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165388 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f102e expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274753:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274760:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274767:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274788:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274796:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274803:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274878:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274884:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274889:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274893:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c3715200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.274896:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274902:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274913:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.274915:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.274918:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274924:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb16538f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274932:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274937:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274943:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274947:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.274950:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274955:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274960:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0xc8545ca3 (ffff9627ffc19200) 00010000:00010000:2.0:1788572550.274962:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.274965:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274970:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165396 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.274988:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165388 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f102e expref: 313 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.275067:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:2.0:1788572550.275070:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572550.275074:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb165388 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f102e expref: 313 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.275081:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715600, rc -2) 00000100:00100000:2.0:1788572550.275148:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+312:2347:x1875453360607488:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2144us (2463us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572550.275158:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 378 00000100:00100000:3.0:1788572550.280725:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.282525:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.282632:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360607872 00000100:00100000:0.0:1788572550.282645:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360607872 00000100:00100000:0.0:1788572550.282712:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1387 00000100:00100000:0.0:1788572550.282722:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+63:2348:x1875453360607872:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572550.282741:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.282763:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000084cbf709/0x85ace67feb16539d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1051 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.282791:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x40:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:0.0:1788572550.282813:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.282818:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x40:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.282827:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000084cbf709/0x85ace67feb16539d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1051 expref: 64 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.282846:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000084cbf709/0x85ace67feb16539d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x40:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1051 expref: 64 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.282859:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a92200, rc 0) 00000100:00100000:0.0:1788572550.282933:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+64:2348:x1875453360607872:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 212us (414us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.282941:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1387 00000100:00100000:2.0:1788572550.285111:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360607744 00000100:00100000:2.0:1788572550.285127:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360607744 00000100:00100000:2.0:1788572550.285200:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4805 00000100:00100000:2.0:1788572550.285211:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+312:35867:x1875453360607744:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.285232:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.285251:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb1653a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f104a expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.285358:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1653ab lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.285373:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1653ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.285384:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1653ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.285395:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7b:0x0]: returning size 3 00000100:00100000:3.0:1788572550.286418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.286474:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.286552:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360608000 00000100:00100000:0.0:1788572550.286560:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360608000 00000100:00100000:0.0:1788572550.286610:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360608128 00000100:00100000:0.0:1788572550.286613:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360608128 00000100:00100000:0.0:1788572550.286643:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1388 00000100:00100000:0.0:1788572550.286652:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+77:2348:x1875453360608000:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572550.286668:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.286686:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000048a4bb5f/0x85ace67feb1653b9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1058 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.286711:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3f:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:0.0:1788572550.286728:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.286733:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.286743:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000048a4bb5f/0x85ace67feb1653b9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1058 expref: 78 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.286759:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000048a4bb5f/0x85ace67feb1653b9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1058 expref: 78 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.286772:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5a93a00, rc 0) 00000100:00100000:0.0:1788572550.286825:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+78:2348:x1875453360608000:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 174us (414us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.286832:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1388 00000100:00100000:0.0:1788572550.286864:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1389 00000100:00100000:0.0:1788572550.286872:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+64:2348:x1875453360608128:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572550.286880:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.286891:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb1653c0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f105f expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.287066:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x3f:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:0.0:1788572550.287076:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3f:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.287080:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.287086:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb1653c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f105f expref: 65 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.287099:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000525813d1/0x85ace67feb1653c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f105f expref: 65 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.287108:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627f5b05600, rc 0) 00000100:00100000:0.0:1788572550.287129:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2348:x1875453360608128:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 259us (656us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.287135:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1389 00010000:00010000:2.0:1788572550.288489:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb1653a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f104a expref: 315 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.288515:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1653ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f104a expref: 314 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.288524:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7b:0x0].0x0 (ffff9627c613b700) 00010000:00010000:2.0:1788572550.288527:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.288530:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497a00, rc 0) 00000100:00100000:2.0:1788572550.288648:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+314:35867:x1875453360607744:12345-192.168.202.40@tcp:101:diff.0 Request processed in 3438us (7933us total) trans 4294970925 rc 0/0 00000100:00100000:2.0:1788572550.288657:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4805 00000100:00100000:3.0:1788572550.296785:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.296909:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360608256 00000100:00100000:2.0:1788572550.296918:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360608256 00000100:00100000:2.0:1788572550.296984:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 359 00000100:00100000:2.0:1788572550.296994:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+314:2347:x1875453360608256:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572550.297036:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572550.297065:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1653ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f104a expref: 314 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.297084:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7b:0x0].0x0 (ffff9627c613b700) 00010000:00010000:2.0:1788572550.297089:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572550.297133:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+313:2347:x1875453360608256:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 141us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.297140:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 359 00000100:00100000:3.0:1788572550.299916:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.300222:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360608384 00000100:00100000:2.0:1788572550.300233:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360608384 00002000:00100000:2.0:1788572550.300286:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x3d:0x0] (0->4095) req@ffff9626c36dc700 x1875453360608384/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.300297:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36dc700 with x1875453360608384 ext(0->4095) 00002000:00010000:2.0:1788572550.300302:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36dc700 00000100:00100000:2.0:1788572550.300327:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 341 00000100:00100000:2.0:1788572550.300336:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2350:x1875453360608384:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.300615:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2350:x1875453360608384:12345-192.168.202.40@tcp:3:diff.0 Request processed in 279us (720us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.300623:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 341 00002000:00100000:2.0:1788572550.300630:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x3d:0x0] (0->4095) req@ffff9626c36dc700 x1875453360608384/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.300637:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36dc700 with x1875453360608384 ext(0->4095) 00002000:00010000:2.0:1788572550.300641:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36dc700 00000100:00100000:1.0:1788572550.305990:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.306133:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360608640 00000100:00100000:2.0:1788572550.306144:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360608640 00000100:00100000:2.0:1788572550.306220:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1033 00000100:00100000:2.0:1788572550.306230:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+313:35867:x1875453360608640:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572550.306697:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+314:35867:x1875453360608640:12345-192.168.202.40@tcp:35:diff.0 Request processed in 467us (715us total) trans 4294970926 rc 0/0 00000100:00100000:2.0:1788572550.306706:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1033 00000100:00100000:3.0:1788572550.313214:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.314711:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360609024 00000100:00100000:1.0:1788572550.314724:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360609024 00000100:00100000:1.0:1788572550.314791:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 379 00000100:00100000:1.0:1788572550.314800:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+314:2348:x1875453360609024:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572550.314873:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.314903:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb1653c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f106d expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.314960:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.314975:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.314987:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.315039:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.315047:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.315053:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315570:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315579:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315585:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315592:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802051200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.315596:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315603:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315617:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:0.0:1788572550.315620:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.315626:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315632:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb1653ce lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315645:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315651:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315656:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315661:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.315664:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315669:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315676:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x2f385ca3 (ffff9627c3e61e00) 00010000:00010000:0.0:1788572550.315678:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.315681:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315686:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53c00/0x85ace67feb1653d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315719:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb1653c7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f106d expref: 315 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315733:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:0.0:1788572550.315736:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572550.315740:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb1653c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f106d expref: 315 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.315749:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802050400, rc -2) 00000100:00100000:0.0:1788572550.316160:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+314:2348:x1875453360609024:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1359us (2953us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572550.316171:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 379 00000100:00100000:2.0:1788572550.316464:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.323095:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360608896 00000100:00100000:3.0:1788572550.323110:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360608896 00000100:00100000:3.0:1788572550.323239:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4806 00000100:00100000:3.0:1788572550.323250:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+314:35867:x1875453360608896:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.323271:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.323294:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1653dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1066 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.323403:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1653e3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.323417:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1653e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.323441:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1653e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.323452:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7a:0x0]: returning size 3 00010000:00010000:3.0:1788572550.323592:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a91c00/0x85ace67feb1653dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1066 expref: 317 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.323613:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1653e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1066 expref: 316 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.323626:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627c613bf00) 00010000:00010000:3.0:1788572550.323628:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572550.323632:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627c9a90a00, rc 0) 00000100:00100000:3.0:1788572550.323709:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+316:35867:x1875453360608896:12345-192.168.202.40@tcp:101:diff.0 Request processed in 461us (7245us total) trans 4294970927 rc 0/0 00000100:00100000:3.0:1788572550.323717:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4806 00000100:00100000:0.0:1788572550.329277:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.329670:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.329938:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360609280 00000100:00100000:3.0:1788572550.329947:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360609280 00000100:00100000:3.0:1788572550.330030:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 360 00000100:00100000:3.0:1788572550.330040:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+316:2347:x1875453360609280:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.330056:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.330079:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627c9a90a00/0x85ace67feb1653e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1066 expref: 316 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.330096:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7a:0x0].0x0 (ffff9627c613bf00) 00010000:00010000:3.0:1788572550.330100:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.330155:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+315:2347:x1875453360609280:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 117us (486us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.330162:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 360 00000100:00100000:2.0:1788572550.330508:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360609152 00000100:00100000:2.0:1788572550.330513:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360609152 00002000:00100000:2.0:1788572550.330543:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x40:0x0] (0->4095) req@ffff9626c36c7b80 x1875453360609152/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.330560:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c7b80 with x1875453360609152 ext(0->4095) 00002000:00010000:2.0:1788572550.330565:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36c7b80 00000100:00100000:2.0:1788572550.330590:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 342 00000100:00100000:2.0:1788572550.330596:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2349:x1875453360609152:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.330796:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2349:x1875453360609152:12345-192.168.202.40@tcp:3:diff.0 Request processed in 200us (1530us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.330802:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 342 00002000:00100000:2.0:1788572550.330808:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x40:0x0] (0->4095) req@ffff9626c36c7b80 x1875453360609152/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.330814:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c36c7b80 with x1875453360609152 ext(0->4095) 00002000:00010000:2.0:1788572550.330817:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c36c7b80 00000100:00100000:0.0:1788572550.336439:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.336535:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360609536 00000100:00100000:1.0:1788572550.336545:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360609536 00000100:00100000:1.0:1788572550.336605:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1034 00000100:00100000:1.0:1788572550.336617:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+315:35867:x1875453360609536:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572550.337473:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+316:35867:x1875453360609536:12345-192.168.202.40@tcp:35:diff.0 Request processed in 854us (1051us total) trans 4294970928 rc 0/0 00000100:00100000:1.0:1788572550.337483:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1034 00000100:00100000:0.0:1788572550.343230:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.343325:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360609920 00000100:00100000:2.0:1788572550.343336:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360609920 00000100:00100000:2.0:1788572550.343392:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4807 00000100:00100000:2.0:1788572550.343402:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+316:35867:x1875453360609920:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.343437:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.343459:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1653f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1123 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.343558:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1653f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x79:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.343572:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1653f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.343587:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1653f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.343598:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x79:0x0]: returning size 3 00010000:00010000:2.0:1788572550.343721:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1653f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1123 expref: 319 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.343738:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1653f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1123 expref: 318 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.343745:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x79:0x0].0x0 (ffff9627ffc19300) 00010000:00010000:2.0:1788572550.343748:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.343752:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497800, rc 0) 00000100:00100000:2.0:1788572550.343814:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+318:35867:x1875453360609920:12345-192.168.202.40@tcp:101:diff.0 Request processed in 413us (593us total) trans 4294970929 rc 0/0 00000100:00100000:2.0:1788572550.343821:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4807 00000100:00100000:3.0:1788572550.348691:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.348781:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.349197:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360610048 00000100:00100000:1.0:1788572550.349208:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360610048 00000100:00100000:1.0:1788572550.349275:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 361 00000100:00100000:1.0:1788572550.349287:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+318:2348:x1875453360610048:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572550.349303:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572550.349328:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1653f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x79:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1123 expref: 318 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.349346:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x79:0x0].0x0 (ffff9627ffc19300) 00010000:00010000:1.0:1788572550.349351:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572550.349413:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+317:2348:x1875453360610048:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 129us (630us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.349420:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 361 00000100:00100000:0.0:1788572550.350310:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360609664 00000100:00100000:0.0:1788572550.350319:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360609664 00000100:00100000:0.0:1788572550.350364:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 380 00000100:00100000:0.0:1788572550.350372:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+317:2347:x1875453360609664:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572550.352649:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.352674:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165406 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f10ac expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.352742:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.352757:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.352773:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.352789:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.352796:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.352802:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353541:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353552:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353558:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353565:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.353570:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353578:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353591:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:0.0:1788572550.353595:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.353600:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353607:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16540d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353621:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353627:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353633:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353638:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.353641:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353647:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353653:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x1dd915a3 (ffff9627fb58d300) 00010000:00010000:0.0:1788572550.353656:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.353658:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353664:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165414 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353702:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165406 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f10ac expref: 319 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353718:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:0.0:1788572550.353720:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572550.353724:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165406 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f10ac expref: 319 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.353735:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc -2) 00000100:00100000:0.0:1788572550.353886:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+317:2347:x1875453360609664:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 3511us (5202us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572550.353905:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 380 00000100:00100000:2.0:1788572550.358722:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.358805:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360610176 00000100:00100000:0.0:1788572550.358816:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360610176 00002000:00100000:0.0:1788572550.358885:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x3f:0x0] (0->4095) req@ffff9626c5430e00 x1875453360610176/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572550.358909:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5430e00 with x1875453360610176 ext(0->4095) 00002000:00010000:0.0:1788572550.358915:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5430e00 00000100:00100000:0.0:1788572550.358951:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 343 00000100:00100000:0.0:1788572550.358961:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+78:2349:x1875453360610176:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572550.359215:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+78:2349:x1875453360610176:12345-192.168.202.40@tcp:3:diff.0 Request processed in 255us (503us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572550.359222:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 343 00002000:00100000:0.0:1788572550.359229:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x3f:0x0] (0->4095) req@ffff9626c5430e00 x1875453360610176/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572550.359237:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5430e00 with x1875453360610176 ext(0->4095) 00002000:00010000:0.0:1788572550.359242:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5430e00 00000100:00100000:2.0:1788572550.368464:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.368566:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360610432 00000100:00100000:3.0:1788572550.368577:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360610432 00000100:00100000:3.0:1788572550.368640:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1035 00000100:00100000:3.0:1788572550.368650:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+317:35867:x1875453360610432:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572550.369106:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+318:35867:x1875453360610432:12345-192.168.202.40@tcp:35:diff.0 Request processed in 456us (653us total) trans 4294970930 rc 0/0 00000100:00100000:3.0:1788572550.369114:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1035 00000100:00100000:3.0:1788572550.376088:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.376120:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.376200:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360610816 00000100:00100000:1.0:1788572550.376208:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360610816 00000100:00100000:1.0:1788572550.376262:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 381 00000100:00100000:1.0:1788572550.376271:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+318:2348:x1875453360610816:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:0.0:1788572550.377394:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360610688 00000100:00100000:0.0:1788572550.377403:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360610688 00000100:00100000:0.0:1788572550.377462:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4808 00000100:00100000:0.0:1788572550.377471:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+320:35867:x1875453360610688:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.377485:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.377506:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16541b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f112a expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.377614:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165422 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x78:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.377629:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165422 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.377641:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165422 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.377653:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x78:0x0]: returning size 3 00010000:00010000:0.0:1788572550.377790:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16541b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f112a expref: 323 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.377810:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f112a expref: 322 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.377820:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x78:0x0].0x0 (ffff9627ffc19600) 00010000:00010000:0.0:1788572550.377823:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.377826:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c1ea3200, rc 0) 00000100:00100000:0.0:1788572550.378101:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+322:35867:x1875453360610688:12345-192.168.202.40@tcp:101:diff.0 Request processed in 630us (2021us total) trans 4294970931 rc 0/0 00000100:00100000:0.0:1788572550.378109:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4808 00010000:00010000:1.0:1788572550.380091:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.380114:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb165430 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1131 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380181:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380197:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380207:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380222:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380229:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380235:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380418:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380443:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380450:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380456:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572550.380462:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380470:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380482:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572550.380485:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572550.380490:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380496:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53e00/0x85ace67feb165437 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380508:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380514:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380520:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380525:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ffd53600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572550.380528:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380534:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380541:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0xa16187a3 (ffff9627c3e61000) 00010000:00010000:1.0:1788572550.380544:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572550.380547:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380553:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16543e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380583:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb165430 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1131 expref: 322 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380593:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:1.0:1788572550.380596:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572550.380599:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb165430 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1131 expref: 322 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.380608:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53a00, rc -2) 00000100:00100000:1.0:1788572550.380674:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+320:2348:x1875453360610816:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 4402us (4551us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572550.380683:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 381 00000100:00100000:0.0:1788572550.385041:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.385504:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360610944 00000100:00100000:3.0:1788572550.385515:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360610944 00000100:00100000:3.0:1788572550.385576:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 362 00000100:00100000:3.0:1788572550.385585:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+320:2349:x1875453360610944:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.385601:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.385625:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165422 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x78:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f112a expref: 320 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.385643:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x78:0x0].0x0 (ffff9627ffc19600) 00010000:00010000:3.0:1788572550.385648:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.385696:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+319:2349:x1875453360610944:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 112us (665us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.385702:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 362 00000100:00100000:0.0:1788572550.387791:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.387901:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360611072 00000100:00100000:1.0:1788572550.387909:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360611072 00002000:00100000:1.0:1788572550.387955:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x3f:0x0] (0->4095) req@ffff9627fb1e2300 x1875453360611072/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.387973:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb1e2300 with x1875453360611072 ext(0->4095) 00002000:00010000:1.0:1788572550.387978:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb1e2300 00000100:00100000:1.0:1788572550.388023:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 344 00000100:00100000:1.0:1788572550.388032:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2350:x1875453360611072:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.391128:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2350:x1875453360611072:12345-192.168.202.40@tcp:3:diff.0 Request processed in 3084us (3333us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.391143:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 344 00002000:00100000:1.0:1788572550.391158:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x3f:0x0] (0->4095) req@ffff9627fb1e2300 x1875453360611072/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.391168:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb1e2300 with x1875453360611072 ext(0->4095) 00002000:00010000:1.0:1788572550.391175:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb1e2300 00000100:00100000:1.0:1788572550.399794:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.399871:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360611328 00000100:00100000:0.0:1788572550.399880:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360611328 00000100:00100000:0.0:1788572550.399945:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1036 00000100:00100000:0.0:1788572550.399955:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+319:35867:x1875453360611328:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.401355:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+320:35867:x1875453360611328:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1393us (1561us total) trans 4294970932 rc 0/0 00000100:00100000:0.0:1788572550.401367:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1036 00000100:00100000:0.0:1788572550.404517:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.404600:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360611456 00000100:00100000:1.0:1788572550.404610:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360611456 00000100:00100000:1.0:1788572550.404707:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4809 00000100:00100000:1.0:1788572550.404719:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+320:35867:x1875453360611456:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572550.404744:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.404765:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb165445 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1218 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.404873:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16544c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.404889:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16544c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.404911:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16544c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572550.404945:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00000004:00200000:1.0:1788572550.404949:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572550.404967:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16544c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.404996:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53a00/0x85ace67feb165445 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1218 expref: 322 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.405034:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ffd53600/0x85ace67feb16544c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1218 expref: 321 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.405043:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572550.405046:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572550.405050:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53600, rc 0) 00000100:00100000:1.0:1788572550.405102:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+321:35867:x1875453360611456:12345-192.168.202.40@tcp:101:diff.0 Request processed in 385us (595us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.405109:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4809 00000100:00100000:0.0:1788572550.409871:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.410168:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360611712 00000100:00100000:3.0:1788572550.410241:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360611712 00000100:00100000:3.0:1788572550.410306:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4810 00000100:00100000:3.0:1788572550.410317:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+321:35867:x1875453360611712:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.410339:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.410359:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb165453 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f121f expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.410468:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6b:0x0]: returning size 16384 00010000:00010000:3.0:1788572550.410600:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb165453 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f121f expref: 323 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.410614:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5f:0x0].0x0 (ffff9627d20cb200) 00010000:00010000:3.0:1788572550.410616:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572550.410621:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb165453 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f121f expref: 323 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.410634:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5000, rc 0) 00000100:00100000:3.0:1788572550.410700:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+322:35867:x1875453360611712:12345-192.168.202.40@tcp:101:diff.0 Request processed in 386us (840us total) trans 4294970933 rc 301/301 00000100:00100000:3.0:1788572550.410707:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4810 00000100:00100000:0.0:1788572550.415201:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.415394:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360611840 00000100:00100000:1.0:1788572550.415405:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360611840 00000100:00100000:1.0:1788572550.415478:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1037 00000100:00100000:1.0:1788572550.415489:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+322:35867:x1875453360611840:12345-192.168.202.40@tcp:37:diff.0 00000100:00100000:0.0:1788572550.421464:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+322:35867:x1875453360611840:12345-192.168.202.40@tcp:37:diff.0 Request processed in 5965us (6262us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.421479:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1037 00000100:00100000:0.0:1788572550.425368:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.425460:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360612096 00000100:00100000:1.0:1788572550.425472:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360612096 00000100:00100000:1.0:1788572550.425538:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1038 00000100:00100000:1.0:1788572550.425551:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec039c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+322:35867:x1875453360612096:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572550.425807:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec039c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+323:35867:x1875453360612096:12345-192.168.202.40@tcp:35:diff.0 Request processed in 259us (449us total) trans 4294970934 rc 0/0 00000100:00100000:1.0:1788572550.425818:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1038 00000100:00100000:3.0:1788572550.429407:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.429513:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360612224 00000100:00100000:2.0:1788572550.429522:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360612224 00000100:00100000:2.0:1788572550.429580:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4811 00000100:00100000:2.0:1788572550.429589:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+323:35867:x1875453360612224:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.429613:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.429632:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb165461 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1226 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.429699:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.429716:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.429729:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.429744:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.429751:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.429757:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430132:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496a00/0x85ace67feb165476 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430141:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496a00/0x85ace67feb165476 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430149:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496a00/0x85ace67feb165476 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.430302:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x75:0x0]: returning size 3 00000004:00200000:2.0:1788572550.430306:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.430328:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496a00/0x85ace67feb165476 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430340:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496a00/0x85ace67feb165476 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430352:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430358:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430364:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430369:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.430374:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430381:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430393:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572550.430396:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.430401:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430409:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165468 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430454:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430461:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430466:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430471:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.430474:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430480:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430487:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x9beb5ca3 (ffff9627ffc19200) 00010000:00010000:2.0:1788572550.430490:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.430493:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430498:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb16546f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x9beb5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430538:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb165461 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1226 expref: 325 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430552:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496a00/0x85ace67feb165476 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1226 expref: 324 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.430559:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627ffc19400) 00010000:00010000:2.0:1788572550.430561:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.430564:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496a00, rc 0) 00000100:00100000:2.0:1788572550.430751:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+324:35867:x1875453360612224:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1160us (1351us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.430759:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4811 00000100:00100000:0.0:1788572550.435415:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.435522:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360612352 00000100:00100000:1.0:1788572550.435531:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360612352 00000100:00100000:1.0:1788572550.435619:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1390 00000100:00100000:1.0:1788572550.435630:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7661850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+78:2350:x1875453360612352:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572550.435648:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.435670:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16547d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f122d expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572550.435700:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x52:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:1.0:1788572550.435724:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=898s v=(0 0) 00010000:00010000:1.0:1788572550.435728:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572550.435738:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16547d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f122d expref: 79 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.435757:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e54cfa1f/0x85ace67feb16547d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f122d expref: 79 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.435770:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53e00, rc 0) 00000100:00100000:1.0:1788572550.435832:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7661850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+79:2350:x1875453360612352:12345-192.168.202.40@tcp:101:diff.0 Request processed in 204us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.435840:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1390 00000100:00100000:2.0:1788572550.440189:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.440340:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360612608 00000100:00100000:3.0:1788572550.440352:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360612608 00000100:00100000:3.0:1788572550.440419:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4812 00000100:00100000:3.0:1788572550.440440:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+324:35867:x1875453360612608:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.440464:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.440486:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165484 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1234 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.440602:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16548b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.440617:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16548b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.440631:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16548b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.440644:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x75:0x0]: returning size 3 00010000:00010000:3.0:1788572550.440782:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165484 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1234 expref: 327 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.440800:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16548b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1234 expref: 326 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.440808:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627ffc19400) 00010000:00010000:3.0:1788572550.440811:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572550.440815:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5800, rc 0) 00000100:00100000:3.0:1788572550.440873:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+326:35867:x1875453360612608:12345-192.168.202.40@tcp:101:diff.0 Request processed in 435us (706us total) trans 4294970935 rc 0/0 00000100:00100000:3.0:1788572550.440881:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4812 00000100:00100000:3.0:1788572550.446694:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.446767:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360612736 00000100:00100000:1.0:1788572550.446777:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360612736 00000100:00100000:1.0:1788572550.446847:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 363 00000100:00100000:1.0:1788572550.446857:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+326:2349:x1875453360612736:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572550.446875:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572550.446910:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16548b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x75:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1234 expref: 326 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.446927:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x75:0x0].0x0 (ffff9627ffc19400) 00010000:00010000:1.0:1788572550.446931:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572550.446987:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+325:2349:x1875453360612736:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 132us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.446994:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 363 00000100:00100000:0.0:1788572550.449328:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.449394:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360612864 00000100:00100000:1.0:1788572550.449404:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360612864 00002000:00100000:1.0:1788572550.449458:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x52:0x0] (0->4095) req@ffff9627fb1e3100 x1875453360612864/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.449469:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb1e3100 with x1875453360612864 ext(0->4095) 00002000:00010000:1.0:1788572550.449475:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb1e3100 00000100:00100000:1.0:1788572550.449506:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 345 00000100:00100000:1.0:1788572550.449515:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+79:2347:x1875453360612864:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.449844:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+79:2347:x1875453360612864:12345-192.168.202.40@tcp:3:diff.0 Request processed in 328us (523us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.449852:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 345 00002000:00100000:1.0:1788572550.449859:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x52:0x0] (0->4095) req@ffff9627fb1e3100 x1875453360612864/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.449875:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb1e3100 with x1875453360612864 ext(0->4095) 00002000:00010000:1.0:1788572550.449878:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb1e3100 00000100:00100000:2.0:1788572550.453983:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.454113:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360613120 00000100:00100000:0.0:1788572550.454124:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360613120 00000100:00100000:0.0:1788572550.454185:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1039 00000100:00100000:0.0:1788572550.454195:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+325:35867:x1875453360613120:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.454615:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+326:35867:x1875453360613120:12345-192.168.202.40@tcp:35:diff.0 Request processed in 419us (639us total) trans 4294970936 rc 0/0 00000100:00100000:0.0:1788572550.454627:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1039 00000100:00100000:3.0:1788572550.457863:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.457964:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360613248 00000100:00100000:2.0:1788572550.457974:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360613248 00000100:00100000:2.0:1788572550.458062:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4813 00000100:00100000:2.0:1788572550.458072:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+326:35867:x1875453360613248:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.458095:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.458117:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165499 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f123b expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458217:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458234:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458244:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458258:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458265:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458271:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458567:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496600/0x85ace67feb1654ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458577:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496600/0x85ace67feb1654ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458585:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496600/0x85ace67feb1654ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.458617:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6c:0x0]: returning size 3 00000004:00200000:2.0:1788572550.458621:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.458641:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496600/0x85ace67feb1654ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458653:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496600/0x85ace67feb1654ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458662:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458668:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458674:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458680:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.458684:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458690:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458702:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572550.458705:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.458710:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458721:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458734:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458740:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458745:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458750:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.458753:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458758:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458765:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x6e9b5ca3 (ffff9627ffc19200) 00010000:00010000:2.0:1788572550.458767:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.458770:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458775:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x6e9b5ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458808:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165499 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f123b expref: 328 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458826:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496600/0x85ace67feb1654ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f123b expref: 327 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.458833:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627ffc19f00) 00010000:00010000:2.0:1788572550.458835:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.458837:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496600, rc 0) 00000100:00100000:2.0:1788572550.458918:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+327:35867:x1875453360613248:12345-192.168.202.40@tcp:101:diff.0 Request processed in 847us (1074us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.458926:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4813 00000100:00100000:2.0:1788572550.463224:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.463361:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360613376 00000100:00100000:0.0:1788572550.463370:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360613376 00000100:00100000:0.0:1788572550.463444:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1391 00000100:00100000:0.0:1788572550.463455:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+65:2347:x1875453360613376:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.463476:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.463497:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092b321b/0x85ace67feb1654b5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1242 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.463524:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x50:0x0] initial LVB size: 3, mtime: 0x6a9b736a, atime: 0x6a9b736a, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:0.0:1788572550.463544:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.463549:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x50:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.463559:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092b321b/0x85ace67feb1654b5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1242 expref: 66 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.463577:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000092b321b/0x85ace67feb1654b5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x50:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1242 expref: 66 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.463590:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af000, rc 0) 00000100:00100000:0.0:1788572550.463656:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2347:x1875453360613376:12345-192.168.202.40@tcp:101:diff.0 Request processed in 202us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.463663:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1391 00000100:00100000:3.0:1788572550.467642:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.467735:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360613632 00000100:00100000:2.0:1788572550.467746:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360613632 00000100:00100000:2.0:1788572550.467812:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4814 00000100:00100000:2.0:1788572550.467822:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+327:35867:x1875453360613632:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.467841:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.467860:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb1654bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1249 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.467968:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.467982:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.467994:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.468043:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6c:0x0]: returning size 3 00010000:00010000:2.0:1788572550.468169:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb1654bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1249 expref: 330 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.468197:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1249 expref: 329 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.468204:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627ffc19f00) 00010000:00010000:2.0:1788572550.468207:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.468210:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497600, rc 0) 00000100:00100000:2.0:1788572550.468273:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+329:35867:x1875453360613632:12345-192.168.202.40@tcp:101:diff.0 Request processed in 452us (641us total) trans 4294970937 rc 0/0 00000100:00100000:2.0:1788572550.468279:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4814 00000100:00100000:3.0:1788572550.473928:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.474041:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360613760 00000100:00100000:2.0:1788572550.474052:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360613760 00000100:00100000:2.0:1788572550.474121:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 364 00000100:00100000:2.0:1788572550.474131:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+329:2347:x1875453360613760:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572550.474147:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572550.474186:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1249 expref: 329 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.474207:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6c:0x0].0x0 (ffff9627ffc19f00) 00010000:00010000:2.0:1788572550.474213:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572550.474298:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+328:2347:x1875453360613760:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 168us (379us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.474305:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 364 00000100:00100000:3.0:1788572550.476292:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.476395:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360613888 00000100:00100000:2.0:1788572550.476404:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360613888 00002000:00100000:2.0:1788572550.476462:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x50:0x0] (0->4095) req@ffff9626c10a1c00 x1875453360613888/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.476475:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a1c00 with x1875453360613888 ext(0->4095) 00002000:00010000:2.0:1788572550.476481:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a1c00 00000100:00100000:2.0:1788572550.476510:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 346 00000100:00100000:2.0:1788572550.476520:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2349:x1875453360613888:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.476759:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2349:x1875453360613888:12345-192.168.202.40@tcp:3:diff.0 Request processed in 240us (475us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.476767:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 346 00002000:00100000:2.0:1788572550.476774:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x50:0x0] (0->4095) req@ffff9626c10a1c00 x1875453360613888/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.476782:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a1c00 with x1875453360613888 ext(0->4095) 00002000:00010000:2.0:1788572550.476785:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a1c00 00000100:00100000:1.0:1788572550.481865:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.481958:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360614144 00000100:00100000:3.0:1788572550.481968:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360614144 00000100:00100000:3.0:1788572550.482064:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1040 00000100:00100000:3.0:1788572550.482074:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+328:35867:x1875453360614144:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:3.0:1788572550.482509:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+329:35867:x1875453360614144:12345-192.168.202.40@tcp:35:diff.0 Request processed in 435us (663us total) trans 4294970938 rc 0/0 00000100:00100000:3.0:1788572550.482518:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1040 00000100:00100000:3.0:1788572550.486249:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.486576:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360614272 00000100:00100000:2.0:1788572550.486587:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360614272 00000100:00100000:2.0:1788572550.486693:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4815 00000100:00100000:2.0:1788572550.486704:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+329:35867:x1875453360614272:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.486730:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.486753:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb1654d1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1250 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.486824:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.486843:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.486920:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.486936:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.486944:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.486950:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487481:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487491:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487499:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.487533:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x74:0x0]: returning size 3 00000004:00200000:2.0:1788572550.487538:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.487559:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487571:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487584:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487590:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487596:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487602:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.487607:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487614:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487627:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572550.487629:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.487636:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487642:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1654d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487653:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487659:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487664:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487669:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.487672:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487677:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487684:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xe44515a3 (ffff9627ffc19200) 00010000:00010000:2.0:1788572550.487687:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.487689:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487695:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1654df lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xe44515a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487728:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb1654d1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1250 expref: 331 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487742:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497600/0x85ace67feb1654e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1250 expref: 330 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.487750:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc19700) 00010000:00010000:2.0:1788572550.487753:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.487755:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497600, rc 0) 00000100:00100000:2.0:1788572550.487829:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+330:35867:x1875453360614272:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1127us (1588us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.487837:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4815 00000100:00100000:0.0:1788572550.491601:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.491690:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360614400 00000100:00100000:1.0:1788572550.491700:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360614400 00000100:00100000:1.0:1788572550.491756:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1392 00000100:00100000:1.0:1788572550.491766:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+66:2349:x1875453360614400:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572550.491785:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.491806:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f5fa93c/0x85ace67feb1654ed lrc: 2/0,0 mode: --/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1257 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572550.491835:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x55:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:1.0:1788572550.491864:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=898s v=(0 0) 00010000:00010000:1.0:1788572550.491868:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x55:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572550.491877:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f5fa93c/0x85ace67feb1654ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1257 expref: 67 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.492041:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001f5fa93c/0x85ace67feb1654ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x55:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1257 expref: 67 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.492057:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53a00, rc 0) 00000100:00100000:1.0:1788572550.492114:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360614400:12345-192.168.202.40@tcp:101:diff.0 Request processed in 349us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.492121:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1392 00000100:00100000:3.0:1788572550.495456:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.495549:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360614656 00000100:00100000:2.0:1788572550.495559:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360614656 00000100:00100000:2.0:1788572550.495659:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4816 00000100:00100000:2.0:1788572550.495670:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+330:35867:x1875453360614656:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.495693:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.495713:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb1654f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f125e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.495846:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1654fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.495861:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1654fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.495873:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1654fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.495884:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x74:0x0]: returning size 3 00010000:00010000:2.0:1788572550.496068:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497c00/0x85ace67feb1654f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f125e expref: 333 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.496087:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1654fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f125e expref: 332 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.496094:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc19700) 00010000:00010000:2.0:1788572550.496097:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.496100:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496000, rc 0) 00000100:00100000:2.0:1788572550.496156:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+332:35867:x1875453360614656:12345-192.168.202.40@tcp:101:diff.0 Request processed in 488us (712us total) trans 4294970939 rc 0/0 00000100:00100000:2.0:1788572550.496163:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4816 00000100:00100000:1.0:1788572550.500233:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.500328:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360614784 00000100:00100000:3.0:1788572550.500338:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360614784 00000100:00100000:3.0:1788572550.500405:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 365 00000100:00100000:3.0:1788572550.500416:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+332:2348:x1875453360614784:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.500445:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.500471:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1654fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x74:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f125e expref: 332 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.500492:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x74:0x0].0x0 (ffff9627ffc19700) 00010000:00010000:3.0:1788572550.500497:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.500555:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+331:2348:x1875453360614784:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 141us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.500563:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 365 00000100:00100000:1.0:1788572550.503596:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.504058:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360614912 00000100:00100000:2.0:1788572550.504069:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360614912 00002000:00100000:2.0:1788572550.504121:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x55:0x0] (0->4095) req@ffff9627e755b800 x1875453360614912/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.504132:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755b800 with x1875453360614912 ext(0->4095) 00002000:00010000:2.0:1788572550.504138:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755b800 00000100:00100000:2.0:1788572550.504164:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 347 00000100:00100000:2.0:1788572550.504184:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360614912:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.504547:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360614912:12345-192.168.202.40@tcp:3:diff.0 Request processed in 373us (958us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.504555:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 347 00002000:00100000:2.0:1788572550.504562:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x55:0x0] (0->4095) req@ffff9627e755b800 x1875453360614912/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.504570:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755b800 with x1875453360614912 ext(0->4095) 00002000:00010000:2.0:1788572550.504574:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755b800 00000100:00100000:3.0:1788572550.513306:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.513601:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360615168 00000100:00100000:1.0:1788572550.513612:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360615168 00000100:00100000:1.0:1788572550.513671:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1041 00000100:00100000:1.0:1788572550.513682:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+331:35867:x1875453360615168:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572550.514136:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+332:35867:x1875453360615168:12345-192.168.202.40@tcp:35:diff.0 Request processed in 454us (839us total) trans 4294970940 rc 0/0 00000100:00100000:1.0:1788572550.514144:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1041 00000100:00100000:2.0:1788572550.522323:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.522480:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360615296 00000100:00100000:3.0:1788572550.522489:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360615296 00000100:00100000:3.0:1788572550.522550:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4817 00000100:00100000:3.0:1788572550.522561:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+332:35867:x1875453360615296:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.522586:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.522608:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165509 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1265 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.522704:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.522720:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.522731:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.522746:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.522753:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.522760:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523088:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5e00/0x85ace67feb16551e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523096:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5e00/0x85ace67feb16551e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523103:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5e00/0x85ace67feb16551e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.523136:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x73:0x0]: returning size 3 00000004:00200000:3.0:1788572550.523140:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572550.523157:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5e00/0x85ace67feb16551e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523169:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5e00/0x85ace67feb16551e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523193:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523200:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523208:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523213:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572550.523217:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523225:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523236:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:3.0:1788572550.523239:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572550.523243:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523249:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165510 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523260:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523265:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523271:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523276:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572550.523278:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523284:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523289:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x2c9ecea3 (ffff9627c5db1a00) 00010000:00010000:3.0:1788572550.523291:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572550.523294:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523299:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165517 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x2c9ecea3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523327:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165509 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1265 expref: 334 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523338:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5e00/0x85ace67feb16551e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1265 expref: 333 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.523345:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627c5db1900) 00010000:00010000:3.0:1788572550.523347:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572550.523349:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5e00, rc 0) 00000100:00100000:3.0:1788572550.523400:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+333:35867:x1875453360615296:12345-192.168.202.40@tcp:101:diff.0 Request processed in 841us (1086us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.523407:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4817 00000100:00100000:0.0:1788572550.530703:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.530800:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360615424 00000100:00100000:2.0:1788572550.530810:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360615424 00000100:00100000:2.0:1788572550.530872:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1393 00000100:00100000:2.0:1788572550.530883:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+79:2350:x1875453360615424:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.530912:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.530932:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ec11391c/0x85ace67feb165525 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f126c expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572550.530959:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x51:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736c, blocks: 0xb 00010000:00010000:2.0:1788572550.530978:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=898s v=(0 0) 00010000:00010000:2.0:1788572550.530983:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572550.530993:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ec11391c/0x85ace67feb165525 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f126c expref: 80 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.531040:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ec11391c/0x85ace67feb165525 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f126c expref: 80 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.531055:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497c00, rc 0) 00000100:00100000:2.0:1788572550.531117:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+80:2350:x1875453360615424:12345-192.168.202.40@tcp:101:diff.0 Request processed in 235us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.531124:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1393 00000100:00100000:1.0:1788572550.545081:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.545290:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360615680 00000100:00100000:2.0:1788572550.545302:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360615680 00000100:00100000:2.0:1788572550.545370:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4818 00000100:00100000:2.0:1788572550.545381:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+333:35867:x1875453360615680:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.545406:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.545444:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb16552c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1273 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.545568:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165533 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.545586:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165533 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.545599:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165533 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.545616:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x73:0x0]: returning size 3 00010000:00010000:2.0:1788572550.545758:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb16552c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1273 expref: 336 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.545777:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165533 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1273 expref: 335 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.545785:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627c5db1900) 00010000:00010000:2.0:1788572550.545788:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.545792:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496e00, rc 0) 00000100:00100000:2.0:1788572550.545858:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+335:35867:x1875453360615680:12345-192.168.202.40@tcp:101:diff.0 Request processed in 478us (786us total) trans 4294970941 rc 0/0 00000100:00100000:2.0:1788572550.545865:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4818 00000100:00100000:3.0:1788572550.559243:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.559324:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360615808 00000100:00100000:1.0:1788572550.559333:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360615808 00002000:00100000:1.0:1788572550.559383:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x51:0x0] (0->4095) req@ffff9626c10a1f80 x1875453360615808/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.559394:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a1f80 with x1875453360615808 ext(0->4095) 00002000:00010000:1.0:1788572550.559399:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c10a1f80 00000100:00100000:1.0:1788572550.559442:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 348 00000100:00100000:1.0:1788572550.559452:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+80:2349:x1875453360615808:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.559705:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+80:2349:x1875453360615808:12345-192.168.202.40@tcp:3:diff.0 Request processed in 253us (470us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.559713:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 348 00002000:00100000:1.0:1788572550.559720:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x51:0x0] (0->4095) req@ffff9626c10a1f80 x1875453360615808/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.559728:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a1f80 with x1875453360615808 ext(0->4095) 00002000:00010000:1.0:1788572550.559732:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c10a1f80 00000100:00100000:0.0:1788572550.564680:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.564731:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.564828:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616064 00000100:00100000:2.0:1788572550.564839:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616064 00000100:00100000:2.0:1788572550.564913:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1042 00000100:00100000:2.0:1788572550.564925:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+335:35867:x1875453360616064:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572550.565369:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+336:35867:x1875453360616064:12345-192.168.202.40@tcp:35:diff.0 Request processed in 445us (698us total) trans 4294970942 rc 0/0 00000100:00100000:2.0:1788572550.565376:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1042 00000100:00100000:1.0:1788572550.565775:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616192 00000100:00100000:1.0:1788572550.565782:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616192 00000100:00100000:1.0:1788572550.565826:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 366 00000100:00100000:1.0:1788572550.565833:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+336:2348:x1875453360616192:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572550.565843:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572550.565867:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165533 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x73:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1273 expref: 336 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.565883:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x73:0x0].0x0 (ffff9627c5db1900) 00010000:00010000:1.0:1788572550.565889:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572550.565936:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+335:2348:x1875453360616192:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 103us (1204us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.565943:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 366 00000100:00100000:0.0:1788572550.569710:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.569815:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616320 00000100:00100000:2.0:1788572550.569825:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616320 00000100:00100000:2.0:1788572550.569904:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4819 00000100:00100000:2.0:1788572550.569914:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+335:35867:x1875453360616320:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.569937:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.569959:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb165541 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f127a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570066:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570082:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570093:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570108:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570115:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570121:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570434:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497e00/0x85ace67feb165556 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570443:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497e00/0x85ace67feb165556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570451:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497e00/0x85ace67feb165556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.570488:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x72:0x0]: returning size 3 00000004:00200000:2.0:1788572550.570492:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.570514:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497e00/0x85ace67feb165556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570526:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497e00/0x85ace67feb165556 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570535:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570542:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570548:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570554:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.570558:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570566:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570577:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572550.570580:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.570585:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570593:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165548 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570606:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570612:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570618:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570623:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.570626:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570632:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570639:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x74f887a3 (ffff9627ffc19000) 00010000:00010000:2.0:1788572550.570643:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.570646:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570652:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb16554f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x74f887a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570685:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb165541 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f127a expref: 337 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570699:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497e00/0x85ace67feb165556 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f127a expref: 336 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.570706:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627ffc19500) 00010000:00010000:2.0:1788572550.570708:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.570710:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497e00, rc 0) 00000100:00100000:2.0:1788572550.570775:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+336:35867:x1875453360616320:12345-192.168.202.40@tcp:101:diff.0 Request processed in 862us (1075us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.570786:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4819 00000100:00100000:3.0:1788572550.578279:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.578638:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616448 00000100:00100000:0.0:1788572550.578650:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616448 00000100:00100000:0.0:1788572550.578714:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1394 00000100:00100000:0.0:1788572550.578726:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+80:2349:x1875453360616448:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.578747:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.578773:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e897e6bb/0x85ace67feb16555d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1281 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.578806:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x53:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:0.0:1788572550.578830:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.578835:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.578845:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e897e6bb/0x85ace67feb16555d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1281 expref: 81 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.578865:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e897e6bb/0x85ace67feb16555d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1281 expref: 81 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.578879:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5afe00, rc 0) 00000100:00100000:0.0:1788572550.578962:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+81:2349:x1875453360616448:12345-192.168.202.40@tcp:101:diff.0 Request processed in 237us (692us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.578969:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1394 00000100:00100000:0.0:1788572550.582925:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.583142:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616704 00000100:00100000:3.0:1788572550.583153:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616704 00000100:00100000:3.0:1788572550.583225:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4820 00000100:00100000:3.0:1788572550.583236:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+336:35867:x1875453360616704:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.583258:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.583281:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb165564 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1288 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.583387:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5a00/0x85ace67feb16556b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.583401:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5a00/0x85ace67feb16556b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.583413:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5a00/0x85ace67feb16556b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572550.583434:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x72:0x0]: returning size 3 00010000:00010000:3.0:1788572550.583562:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb165564 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1288 expref: 339 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.583579:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5a00/0x85ace67feb16556b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1288 expref: 338 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.583589:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627ffc19500) 00010000:00010000:3.0:1788572550.583592:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572550.583596:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5a00, rc 0) 00000100:00100000:3.0:1788572550.583660:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+338:35867:x1875453360616704:12345-192.168.202.40@tcp:101:diff.0 Request processed in 426us (745us total) trans 4294970943 rc 0/0 00000100:00100000:3.0:1788572550.583668:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4820 00000100:00100000:3.0:1788572550.591038:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.591128:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616832 00000100:00100000:0.0:1788572550.591139:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616832 00002000:00100000:0.0:1788572550.591200:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x53:0x0] (0->4095) req@ffff9626c10a0380 x1875453360616832/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572550.591212:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a0380 with x1875453360616832 ext(0->4095) 00002000:00010000:0.0:1788572550.591218:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c10a0380 00000100:00100000:0.0:1788572550.591253:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 349 00000100:00100000:0.0:1788572550.591263:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+81:2349:x1875453360616832:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572550.591538:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+81:2349:x1875453360616832:12345-192.168.202.40@tcp:3:diff.0 Request processed in 275us (539us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572550.591546:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 349 00002000:00100000:0.0:1788572550.591553:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x53:0x0] (0->4095) req@ffff9626c10a0380 x1875453360616832/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572550.591561:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a0380 with x1875453360616832 ext(0->4095) 00002000:00010000:0.0:1788572550.591564:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c10a0380 00000100:00100000:1.0:1788572550.597940:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.598051:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360616960 00000100:00100000:3.0:1788572550.598061:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360616960 00000100:00100000:3.0:1788572550.598134:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 367 00000100:00100000:3.0:1788572550.598143:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+338:2349:x1875453360616960:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.598159:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.598190:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5a00/0x85ace67feb16556b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x72:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1288 expref: 338 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.598208:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x72:0x0].0x0 (ffff9627ffc19500) 00010000:00010000:3.0:1788572550.598214:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.598268:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+337:2349:x1875453360616960:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 127us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.598275:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 367 00000100:00100000:3.0:1788572550.605216:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.605449:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360617216 00000100:00100000:0.0:1788572550.605459:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360617216 00000100:00100000:0.0:1788572550.605523:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1043 00000100:00100000:0.0:1788572550.605534:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+337:35867:x1875453360617216:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.606154:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+338:35867:x1875453360617216:12345-192.168.202.40@tcp:35:diff.0 Request processed in 620us (948us total) trans 4294970944 rc 0/0 00000100:00100000:0.0:1788572550.606162:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1043 00000100:00100000:1.0:1788572550.613745:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.614126:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360617344 00000100:00100000:2.0:1788572550.614137:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360617344 00000100:00100000:2.0:1788572550.614216:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4821 00000100:00100000:2.0:1788572550.614227:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+338:35867:x1875453360617344:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.614253:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.614275:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb165579 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f128f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614367:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614383:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614394:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614410:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614418:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614439:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614725:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497400/0x85ace67feb16558e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614738:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497400/0x85ace67feb16558e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614744:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497400/0x85ace67feb16558e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.614774:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x71:0x0]: returning size 3 00000004:00200000:2.0:1788572550.614778:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572550.614797:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497400/0x85ace67feb16558e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614808:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497400/0x85ace67feb16558e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614817:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614823:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614830:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614836:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.614840:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614860:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614873:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:2.0:1788572550.614877:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.614883:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.614890:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496200/0x85ace67feb165580 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615071:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615078:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615084:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615089:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572550.615092:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615099:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615106:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xbd5240a3 (ffff9627ffc19000) 00010000:00010000:2.0:1788572550.615109:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572550.615112:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615118:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165587 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xbd5240a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615152:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb165579 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f128f expref: 340 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615165:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497400/0x85ace67feb16558e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f128f expref: 339 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.615312:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627ffc19100) 00010000:00010000:2.0:1788572550.615315:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.615317:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497400, rc 0) 00000100:00100000:2.0:1788572550.615378:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+339:35867:x1875453360617344:12345-192.168.202.40@tcp:101:diff.0 Request processed in 1153us (1643us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.615386:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4821 00000100:00100000:1.0:1788572550.623861:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.623977:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360617472 00000100:00100000:3.0:1788572550.623987:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360617472 00000100:00100000:3.0:1788572550.624074:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1395 00000100:00100000:3.0:1788572550.624084:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+67:2349:x1875453360617472:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:3.0:1788572550.624102:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.624122:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e713ae5/0x85ace67feb165595 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1296 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572550.624149:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x54:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:3.0:1788572550.624184:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=898s v=(0 0) 00010000:00010000:3.0:1788572550.624190:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572550.624200:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e713ae5/0x85ace67feb165595 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1296 expref: 68 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.624219:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e713ae5/0x85ace67feb165595 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1296 expref: 68 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.624233:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5a00, rc 0) 00000100:00100000:3.0:1788572550.624299:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2349:x1875453360617472:12345-192.168.202.40@tcp:101:diff.0 Request processed in 216us (461us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.624306:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1395 00000100:00100000:0.0:1788572550.635678:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.635779:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360617728 00000100:00100000:2.0:1788572550.635789:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360617728 00000100:00100000:2.0:1788572550.635850:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4822 00000100:00100000:2.0:1788572550.635860:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+339:35867:x1875453360617728:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.635880:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.635909:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb16559c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f129d expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.636059:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1655a3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.636073:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1655a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.636084:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1655a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.636094:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x71:0x0]: returning size 3 00010000:00010000:2.0:1788572550.636239:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb16559c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f129d expref: 342 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.636258:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1655a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f129d expref: 341 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.636267:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627ffc19100) 00010000:00010000:2.0:1788572550.636270:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.636274:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496c00, rc 0) 00000100:00100000:2.0:1788572550.636409:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+341:35867:x1875453360617728:12345-192.168.202.40@tcp:101:diff.0 Request processed in 550us (740us total) trans 4294970945 rc 0/0 00000100:00100000:2.0:1788572550.636417:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4822 00000100:00100000:1.0:1788572550.653703:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.653759:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.653938:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360617856 00000100:00100000:3.0:1788572550.653949:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360617856 00000100:00100000:3.0:1788572550.654059:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 368 00000100:00100000:3.0:1788572550.654071:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+341:2347:x1875453360617856:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.654089:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.654113:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1655a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x71:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f129d expref: 341 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.654133:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x71:0x0].0x0 (ffff9627ffc19100) 00010000:00010000:3.0:1788572550.654138:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.654198:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+340:2347:x1875453360617856:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 129us (505us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.654205:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 368 00000100:00100000:2.0:1788572550.657520:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360617984 00000100:00100000:2.0:1788572550.657537:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360617984 00002000:00100000:2.0:1788572550.657600:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x54:0x0] (0->4095) req@ffff9627e755a300 x1875453360617984/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.657613:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755a300 with x1875453360617984 ext(0->4095) 00002000:00010000:2.0:1788572550.657619:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755a300 00000100:00100000:2.0:1788572550.657712:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 350 00000100:00100000:2.0:1788572550.657721:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2349:x1875453360617984:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.657969:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2349:x1875453360617984:12345-192.168.202.40@tcp:3:diff.0 Request processed in 248us (4207us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.657976:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 350 00002000:00100000:2.0:1788572550.657983:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x54:0x0] (0->4095) req@ffff9627e755a300 x1875453360617984/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.657990:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755a300 with x1875453360617984 ext(0->4095) 00002000:00010000:2.0:1788572550.657994:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755a300 00000100:00100000:1.0:1788572550.662576:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.662689:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360618240 00000100:00100000:0.0:1788572550.662700:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360618240 00000100:00100000:0.0:1788572550.662766:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1044 00000100:00100000:0.0:1788572550.662776:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec038c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+340:35867:x1875453360618240:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.663463:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec038c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+341:35867:x1875453360618240:12345-192.168.202.40@tcp:35:diff.0 Request processed in 685us (894us total) trans 4294970946 rc 0/0 00000100:00100000:0.0:1788572550.663473:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1044 00000100:00100000:3.0:1788572550.666501:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.667089:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360618368 00000100:00100000:0.0:1788572550.667100:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360618368 00000100:00100000:0.0:1788572550.667168:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4823 00000100:00100000:0.0:1788572550.667186:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+341:35867:x1875453360618368:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:0.0:1788572550.667209:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.667230:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1655b1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12a4 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667293:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667308:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667319:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667332:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667339:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667345:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667618:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae000/0x85ace67feb1655c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667628:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae000/0x85ace67feb1655c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667635:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae000/0x85ace67feb1655c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.667667:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x70:0x0]: returning size 3 00000004:00200000:0.0:1788572550.667671:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572550.667689:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae000/0x85ace67feb1655c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667700:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae000/0x85ace67feb1655c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667709:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667715:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667722:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667727:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5afc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.667732:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667739:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667749:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.667753:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.667757:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667764:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667775:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667781:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667786:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667791:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.667794:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667800:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667805:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x5abf9a3 (ffff9627fb58df00) 00010000:00010000:0.0:1788572550.667808:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.667810:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667815:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x5abf9a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667840:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1655b1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12a4 expref: 343 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667857:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae000/0x85ace67feb1655c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12a4 expref: 342 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.667864:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:0.0:1788572550.667866:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.667869:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae000, rc 0) 00000100:00100000:0.0:1788572550.668045:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+342:35867:x1875453360618368:12345-192.168.202.40@tcp:101:diff.0 Request processed in 816us (1507us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.668054:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4823 00000100:00100000:3.0:1788572550.673161:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.673261:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360618496 00000100:00100000:0.0:1788572550.673271:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360618496 00000100:00100000:0.0:1788572550.673336:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 382 00000100:00100000:0.0:1788572550.673345:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c97ce050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+342:2349:x1875453360618496:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572550.673383:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.673402:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1655cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12ab expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673474:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673489:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673500:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673516:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673525:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673531:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673830:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae200/0x85ace67feb1655e2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673839:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae200/0x85ace67feb1655e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673852:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae200/0x85ace67feb1655e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.673873:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6f:0x0]: returning size 3 00000004:00200000:0.0:1788572550.673877:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572550.673904:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae200/0x85ace67feb1655e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673916:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae200/0x85ace67feb1655e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673925:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673931:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673938:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673949:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.673953:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673960:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673970:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.673973:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.673977:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673983:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1655d4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673994:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.673999:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674042:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674047:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5afc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.674049:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674055:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674066:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x4e05b2a3 (ffff9627fb58df00) 00010000:00010000:0.0:1788572550.674069:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.674071:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674077:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1655db lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x4e05b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674105:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1655cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ab expref: 344 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674119:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae200/0x85ace67feb1655e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ab expref: 343 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674126:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627c5160600) 00010000:00010000:0.0:1788572550.674129:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.674131:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae200, rc 0) 00010000:00010000:0.0:1788572550.674138:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.674145:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1655e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12b2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674169:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674177:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674183:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674196:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674202:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674207:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674357:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afa00/0x85ace67feb1655fe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674365:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afa00/0x85ace67feb1655fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674371:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afa00/0x85ace67feb1655fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.674385:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6e:0x0]: returning size 3 00000004:00200000:0.0:1788572550.674388:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572550.674397:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afa00/0x85ace67feb1655fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674406:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afa00/0x85ace67feb1655fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674412:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674418:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674442:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674447:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.674450:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674455:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674463:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.674465:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.674468:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674474:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1655f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674482:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674489:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674494:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674499:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.674502:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674512:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674519:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x965f6ba3 (ffff9627c5160200) 00010000:00010000:0.0:1788572550.674521:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.674524:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674529:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1655f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x965f6ba3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674550:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1655e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12b2 expref: 345 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674560:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afa00/0x85ace67feb1655fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12b2 expref: 344 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674567:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c5160f00) 00010000:00010000:0.0:1788572550.674569:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.674572:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5afa00, rc 0) 00010000:00010000:0.0:1788572550.674576:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.674584:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165605 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12b9 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674608:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674617:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674628:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674640:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674646:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674652:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674816:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aee00/0x85ace67feb16561a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674826:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aee00/0x85ace67feb16561a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674832:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aee00/0x85ace67feb16561a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572550.674845:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6d:0x0]: returning size 3 00000004:00200000:0.0:1788572550.674849:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572550.674860:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aee00/0x85ace67feb16561a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674868:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aee00/0x85ace67feb16561a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674875:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674884:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674889:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674894:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.674897:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674903:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674912:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.674914:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.674917:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674922:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16560c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674931:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674937:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674943:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674948:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.674951:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674957:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674964:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xdeb924a3 (ffff9627c5160700) 00010000:00010000:0.0:1788572550.674966:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.674969:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674975:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af600/0x85ace67feb165613 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xdeb924a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.674997:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165605 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12b9 expref: 346 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675058:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aee00/0x85ace67feb16561a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12b9 expref: 345 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675064:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627c5160e00) 00010000:00010000:0.0:1788572550.675067:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572550.675069:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aee00, rc 0) 00010000:00010000:0.0:1788572550.675075:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.675081:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165621 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12c0 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675105:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675113:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675119:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675130:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675136:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675143:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675225:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675232:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675238:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675243:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea3200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.675246:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675252:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675260:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.675262:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.675265:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675271:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea3200/0x85ace67feb165628 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675280:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675285:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675291:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675296:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c1ea2a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.675299:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675305:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675311:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xc8545ca3 (ffff9627c5160000) 00010000:00010000:0.0:1788572550.675314:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.675316:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675322:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c1ea2a00/0x85ace67feb16562f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xc8545ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675341:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165621 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12c0 expref: 346 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675354:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.675356:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572550.675359:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165621 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12c0 expref: 346 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.675366:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aec00, rc -2) 00000100:00100000:0.0:1788572550.675449:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c97ce050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+345:2349:x1875453360618496:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2104us (2297us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572550.675459:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 382 00000100:00100000:3.0:1788572550.677696:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.678060:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360618624 00000100:00100000:2.0:1788572550.678071:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360618624 00000100:00100000:2.0:1788572550.678126:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1396 00000100:00100000:2.0:1788572550.678137:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+68:2348:x1875453360618624:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.678156:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.678191:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f499556/0x85ace67feb165636 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12dc expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572550.678220:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x53:0x0] initial LVB size: 3, mtime: 0x6a9b7369, atime: 0x6a9b7369, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:2.0:1788572550.678244:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=898s v=(0 0) 00010000:00010000:2.0:1788572550.678250:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x53:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572550.678260:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f499556/0x85ace67feb165636 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12dc expref: 69 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.678277:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f499556/0x85ace67feb165636 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x53:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12dc expref: 69 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.678292:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496200, rc 0) 00000100:00100000:2.0:1788572550.678358:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+69:2348:x1875453360618624:12345-192.168.202.40@tcp:101:diff.0 Request processed in 222us (669us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.678365:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1396 00000100:00100000:3.0:1788572550.689802:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.689856:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.689883:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.689918:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360619008 00000100:00100000:3.0:1788572550.689926:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360619008 00000100:00100000:3.0:1788572550.689972:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360619136 00000100:00100000:3.0:1788572550.689975:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360619136 00000100:00100000:3.0:1788572550.690000:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1397 00000100:00100000:3.0:1788572550.690034:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+69:2347:x1875453360619008:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:3.0:1788572550.690053:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572550.690074:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004070a610/0x85ace67feb16563d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12ea expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572550.690101:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x51:0x0] initial LVB size: 3, mtime: 0x6a9b736a, atime: 0x6a9b736a, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:3.0:1788572550.690122:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=898s v=(0 0) 00010000:00010000:3.0:1788572550.690127:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x51:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572550.690136:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004070a610/0x85ace67feb16563d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ea expref: 70 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.690154:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004070a610/0x85ace67feb16563d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x51:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ea expref: 70 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.690167:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4c00, rc 0) 00000100:00100000:0.0:1788572550.690321:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1398 00000100:00100000:0.0:1788572550.690333:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+81:2347:x1875453360619136:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:0.0:1788572550.690341:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.690352:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dfba78e7/0x85ace67feb165644 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12f1 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572550.690369:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x54:0x0] initial LVB size: 3, mtime: 0x6a9b736a, atime: 0x6a9b736a, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:0.0:1788572550.690380:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=898s v=(0 0) 00010000:00010000:0.0:1788572550.690383:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x54:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572550.690389:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dfba78e7/0x85ace67feb165644 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12f1 expref: 82 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.690402:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dfba78e7/0x85ace67feb165644 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x54:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12f1 expref: 82 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.690415:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af600, rc 0) 00000100:00100000:0.0:1788572550.690484:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2347:x1875453360619136:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 154us (599us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.690490:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1398 00000100:00100000:1.0:1788572550.690565:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360618880 00000100:00100000:1.0:1788572550.690570:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360618880 00000100:00100000:1.0:1788572550.690614:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4824 00000100:00100000:1.0:1788572550.690623:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+345:35867:x1875453360618880:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572550.690635:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.690649:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16564b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12e3 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.690745:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165652 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.690759:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165652 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.690770:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165652 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572550.690781:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x70:0x0]: returning size 3 00010000:00010000:1.0:1788572550.690922:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb16564b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12e3 expref: 348 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.690940:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165652 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x70:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12e3 expref: 347 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.690949:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:1.0:1788572550.690951:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572550.690954:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc 0) 00000100:00100000:1.0:1788572550.690976:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+347:35867:x1875453360618880:12345-192.168.202.40@tcp:101:diff.0 Request processed in 355us (1184us total) trans 4294970947 rc 0/0 00000100:00100000:1.0:1788572550.690982:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4824 00000100:00100000:3.0:1788572550.691445:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+70:2347:x1875453360619008:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 1410us (1596us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.691457:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1397 00000100:00100000:3.0:1788572550.691515:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.694579:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360619264 00000100:00100000:1.0:1788572550.694595:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360619264 00000100:00100000:1.0:1788572550.694668:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1399 00000100:00100000:1.0:1788572550.694679:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+70:2347:x1875453360619264:12345-192.168.202.40@tcp:101:ll_agl.0 00010000:00010000:1.0:1788572550.694698:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.694723:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb165660 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12f8 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572550.694758:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x52:0x0] initial LVB size: 3, mtime: 0x6a9b736a, atime: 0x6a9b736a, ctime: 0x6a9b736b, blocks: 0xb 00010000:00010000:1.0:1788572550.694777:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=898s v=(0 0) 00010000:00010000:1.0:1788572550.694782:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x52:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572550.694792:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb165660 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12f8 expref: 71 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.694810:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b5a71f79/0x85ace67feb165660 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x52:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12f8 expref: 71 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.694824:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ffd53c00, rc 0) 00000100:00100000:1.0:1788572550.694889:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2347:x1875453360619264:12345-192.168.202.40@tcp:101:ll_agl.0 Request processed in 212us (3373us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.694903:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1399 00000100:00100000:2.0:1788572550.699339:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.699674:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.699883:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360619520 00000100:00100000:2.0:1788572550.699903:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360619520 00002000:00100000:2.0:1788572550.699952:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x53:0x0] (0->4095) req@ffff9627e755aa00 x1875453360619520/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.699969:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755aa00 with x1875453360619520 ext(0->4095) 00002000:00010000:2.0:1788572550.699975:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755aa00 00000100:00100000:2.0:1788572550.700030:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 351 00000100:00100000:2.0:1788572550.700039:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2350:x1875453360619520:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.700282:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2350:x1875453360619520:12345-192.168.202.40@tcp:3:diff.0 Request processed in 243us (608us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.700291:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 351 00002000:00100000:2.0:1788572550.700299:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x53:0x0] (0->4095) req@ffff9627e755aa00 x1875453360619520/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.700307:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755aa00 with x1875453360619520 ext(0->4095) 00002000:00010000:2.0:1788572550.700311:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755aa00 00000100:00100000:3.0:1788572550.702121:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360619392 00000100:00100000:3.0:1788572550.702135:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360619392 00000100:00100000:3.0:1788572550.702215:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 369 00000100:00100000:3.0:1788572550.702226:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+347:2348:x1875453360619392:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.702243:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.702270:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165652 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x70:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12e3 expref: 347 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.702292:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x70:0x0].0x0 (ffff9627c29dad00) 00010000:00010000:3.0:1788572550.702298:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.702358:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+346:2348:x1875453360619392:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 133us (3027us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.702366:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 369 00000100:00100000:1.0:1788572550.718801:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.718921:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360619776 00000100:00100000:0.0:1788572550.718932:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360619776 00000100:00100000:0.0:1788572550.718991:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1045 00000100:00100000:0.0:1788572550.719001:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+346:35867:x1875453360619776:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.719436:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+347:35867:x1875453360619776:12345-192.168.202.40@tcp:35:diff.0 Request processed in 423us (632us total) trans 4294970948 rc 0/0 00000100:00100000:0.0:1788572550.719445:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1045 00000100:00100000:0.0:1788572550.728068:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.728308:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620032 00000100:00100000:2.0:1788572550.728320:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620032 00000100:00100000:2.0:1788572550.728390:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4825 00000100:00100000:2.0:1788572550.728401:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+347:35867:x1875453360620032:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.728436:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.728459:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165667 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f12ff expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.728590:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb16566e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.728607:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb16566e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.728624:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb16566e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.728636:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6f:0x0]: returning size 3 00010000:00010000:2.0:1788572550.728772:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165667 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ff expref: 350 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.728792:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb16566e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ff expref: 349 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.728801:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627c5160600) 00010000:00010000:2.0:1788572550.728804:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.728809:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497800, rc 0) 00000100:00100000:2.0:1788572550.728904:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+349:35867:x1875453360620032:12345-192.168.202.40@tcp:101:diff.0 Request processed in 505us (847us total) trans 4294970949 rc 0/0 00000100:00100000:2.0:1788572550.728912:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4825 00000100:00100000:3.0:1788572550.733694:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.734113:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620160 00000100:00100000:0.0:1788572550.734125:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620160 00000100:00100000:0.0:1788572550.734216:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 370 00000100:00100000:0.0:1788572550.734226:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+349:2349:x1875453360620160:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572550.734243:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572550.734265:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb16566e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f12ff expref: 349 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.734282:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6f:0x0].0x0 (ffff9627c5160600) 00010000:00010000:0.0:1788572550.734286:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572550.734342:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+348:2349:x1875453360620160:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 117us (658us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.734352:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 370 00000100:00100000:1.0:1788572550.737847:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.738143:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620288 00000100:00100000:0.0:1788572550.738155:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620288 00002000:00100000:0.0:1788572550.738219:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x51:0x0] (0->4095) req@ffff9627e755a680 x1875453360620288/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572550.738231:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755a680 with x1875453360620288 ext(0->4095) 00002000:00010000:0.0:1788572550.738237:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755a680 00000100:00100000:0.0:1788572550.738260:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 352 00000100:00100000:0.0:1788572550.738269:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2347:x1875453360620288:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:0.0:1788572550.740338:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2347:x1875453360620288:12345-192.168.202.40@tcp:3:diff.0 Request processed in 2059us (2491us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788572550.740355:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 352 00002000:00100000:0.0:1788572550.740370:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x51:0x0] (0->4095) req@ffff9627e755a680 x1875453360620288/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572550.740381:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e755a680 with x1875453360620288 ext(0->4095) 00002000:00010000:0.0:1788572550.740387:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e755a680 00000100:00100000:3.0:1788572550.752564:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.757564:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620416 00000100:00100000:1.0:1788572550.757585:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620416 00000100:00100000:1.0:1788572550.757868:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 383 00000100:00100000:1.0:1788572550.757881:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+348:2350:x1875453360620416:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572550.757968:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.757994:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16567c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1306 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758084:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758103:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758116:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758134:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758142:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758149:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758530:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758539:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758546:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758552:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572550.758556:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758564:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758578:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572550.758582:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572550.758588:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758596:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165683 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758609:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758615:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758620:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758626:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572550.758628:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758634:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758642:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x2f385ca3 (ffff9627c3e61e00) 00010000:00010000:1.0:1788572550.758644:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572550.758647:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758652:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb16568a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x2f385ca3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758685:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16567c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1306 expref: 349 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758696:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572550.758698:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572550.758702:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb16567c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1306 expref: 349 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.758712:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc -2) 00000100:00100000:1.0:1788572550.759924:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+348:2350:x1875453360620416:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2035us (7362us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572550.759938:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 383 00000100:00100000:2.0:1788572550.760983:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.761261:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620672 00000100:00100000:1.0:1788572550.761271:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620672 00000100:00100000:1.0:1788572550.761330:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1046 00000100:00100000:1.0:1788572550.761340:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+348:35867:x1875453360620672:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:1.0:1788572550.761973:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+349:35867:x1875453360620672:12345-192.168.202.40@tcp:35:diff.0 Request processed in 632us (992us total) trans 4294970950 rc 0/0 00000100:00100000:1.0:1788572550.761983:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1046 00000100:00100000:0.0:1788572550.779259:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.779340:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.779373:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.779403:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360621184 00000100:00100000:0.0:1788572550.779413:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360621184 00000100:00100000:0.0:1788572550.779489:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 600 00000100:00100000:0.0:1788572550.779502:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360621184:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572550.779580:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360621184:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 81us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572550.779588:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 600 00000100:00080000:0.0:1788572550.779610:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788572550.779650:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.779999:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360621056 00000100:00100000:3.0:1788572550.780032:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360621056 00000100:00100000:3.0:1788572550.780072:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 601 00000100:00100000:3.0:1788572550.780081:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360621056:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572550.780120:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360621056:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 41us (470us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.780128:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 601 00000100:00100000:3.0:1788572550.780154:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620928 00000100:00100000:3.0:1788572550.780159:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620928 00000100:00100000:3.0:1788572550.780203:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 635 00000100:00100000:3.0:1788572550.780210:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360620928:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572550.780247:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2350:x1875453360620928:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 38us (907us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.780254:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 635 00000100:00080000:0.0:1788572550.780352:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1788572550.780356:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572550.782159:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360620800 00000100:00100000:2.0:1788572550.782181:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360620800 00000100:00100000:2.0:1788572550.782238:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 681 00000100:00100000:2.0:1788572550.782248:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2349:x1875453360620800:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572550.782305:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2349:x1875453360620800:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 58us (3055us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572550.782313:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 681 00000100:00100000:2.0:1788572550.786141:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.786677:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360621440 00000100:00100000:1.0:1788572550.786687:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360621440 00000100:00100000:1.0:1788572550.786746:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4826 00000100:00100000:1.0:1788572550.786756:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+349:35867:x1875453360621440:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:1.0:1788572550.786781:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572550.786804:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165691 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1345 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.804344:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165698 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.804373:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165698 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.804386:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165698 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572550.804401:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6e:0x0]: returning size 3 00010000:00010000:1.0:1788572550.804645:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165691 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1345 expref: 352 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.804666:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165698 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1345 expref: 351 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.804675:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c5160f00) 00010000:00010000:1.0:1788572550.804678:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572550.804682:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc 0) 00000100:00100000:2.0:1788572550.806139:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.807314:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360621568 00000100:00100000:0.0:1788572550.807359:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360621568 00000100:00100000:0.0:1788572550.810913:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 384 00000100:00100000:0.0:1788572550.810961:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+352:2349:x1875453360621568:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:1.0:1788572550.811353:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+353:35867:x1875453360621440:12345-192.168.202.40@tcp:101:diff.0 Request processed in 24595us (25217us total) trans 4294970951 rc 0/0 00000100:00100000:1.0:1788572550.811362:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4826 00010000:00010000:0.0:1788572550.821475:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.821502:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1656a6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f134c expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821569:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821584:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821596:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821612:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821619:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821626:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821820:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821828:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821834:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821840:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.821845:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821852:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821864:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.821867:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.821872:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.821878:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656ad lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822040:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822048:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822054:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822059:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.822063:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822069:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822078:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x1dd915a3 (ffff9627c5160000) 00010000:00010000:0.0:1788572550.822080:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.822083:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822088:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x1dd915a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822123:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1656a6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f134c expref: 353 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822133:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.822135:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572550.822138:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1656a6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f134c expref: 353 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.822146:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af400, rc -2) 00000100:00100000:0.0:1788572550.823112:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+351:2349:x1875453360621568:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 12171us (16974us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572550.823126:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 384 00000100:00100000:3.0:1788572550.833740:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.833953:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360621696 00000100:00100000:1.0:1788572550.833964:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360621696 00000100:00100000:1.0:1788572550.834062:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 371 00000100:00100000:1.0:1788572550.834073:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+351:2347:x1875453360621696:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572550.834090:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572550.834114:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165698 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1345 expref: 351 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572550.834140:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6e:0x0].0x0 (ffff9627c5160f00) 00010000:00010000:1.0:1788572550.834145:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572550.834212:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+350:2347:x1875453360621696:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 140us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572550.834220:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 371 00000100:00100000:3.0:1788572550.854274:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.854371:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360621824 00000100:00100000:1.0:1788572550.854382:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360621824 00002000:00100000:1.0:1788572550.854464:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x54:0x0] (0->4095) req@ffff9627d131d500 x1875453360621824/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.854475:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131d500 with x1875453360621824 ext(0->4095) 00002000:00010000:1.0:1788572550.854481:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627d131d500 00000100:00100000:1.0:1788572550.854514:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 353 00000100:00100000:1.0:1788572550.854524:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2349:x1875453360621824:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:1.0:1788572550.854770:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2349:x1875453360621824:12345-192.168.202.40@tcp:3:diff.0 Request processed in 247us (506us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788572550.854778:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 353 00002000:00100000:1.0:1788572550.854784:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x54:0x0] (0->4095) req@ffff9627d131d500 x1875453360621824/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572550.854791:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131d500 with x1875453360621824 ext(0->4095) 00002000:00010000:1.0:1788572550.854795:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627d131d500 00000100:00100000:3.0:1788572550.873119:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.873220:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360622080 00000100:00100000:2.0:1788572550.873231:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360622080 00000100:00100000:2.0:1788572550.873292:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1047 00000100:00100000:2.0:1788572550.873302:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+350:35867:x1875453360622080:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:0.0:1788572550.874456:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+351:35867:x1875453360622080:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1149us (1339us total) trans 4294970952 rc 0/0 00000100:00100000:0.0:1788572550.874467:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1047 00000100:00100000:0.0:1788572550.883164:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572550.884537:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360622208 00000100:00100000:1.0:1788572550.884550:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360622208 00000100:00100000:1.0:1788572550.884618:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 385 00000100:00100000:1.0:1788572550.884629:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+351:2350:x1875453360622208:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:3.0:1788572550.885059:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572550.885294:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360622464 00000100:00100000:2.0:1788572550.885301:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360622464 00000100:00100000:2.0:1788572550.885346:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4827 00000100:00100000:2.0:1788572550.885354:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+353:35867:x1875453360622464:12345-192.168.202.40@tcp:101:diff.0 00010000:00010000:2.0:1788572550.885366:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572550.885388:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1656bb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14aa expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.885506:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656c2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.885522:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.885533:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572550.885544:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x6d:0x0]: returning size 3 00010000:00010000:2.0:1788572550.885679:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1656bb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14aa expref: 356 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.885698:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656c2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14aa expref: 355 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572550.885707:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627c5160e00) 00010000:00010000:2.0:1788572550.885710:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572550.885714:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496c00, rc 0) 00000100:00100000:2.0:1788572550.885767:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+355:35867:x1875453360622464:12345-192.168.202.40@tcp:101:diff.0 Request processed in 413us (709us total) trans 4294970953 rc 0/0 00000100:00100000:2.0:1788572550.885774:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4827 00010000:00010000:0.0:1788572550.889249:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572550.889279:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656d0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f13c3 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889351:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889366:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889379:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889393:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889401:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889408:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889618:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889627:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889633:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889640:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.889645:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889653:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889665:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.889668:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.889673:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889680:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1656d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889691:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889697:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889703:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889708:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572550.889710:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889716:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889726:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0xa16187a3 (ffff9627c5160000) 00010000:00010000:0.0:1788572550.889729:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572550.889731:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889737:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb1656de lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0xa16187a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889769:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656d0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f13c3 expref: 355 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889780:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:0.0:1788572550.889782:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572550.889786:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1656d0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f13c3 expref: 355 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572550.889795:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af800, rc -2) 00000100:00100000:0.0:1788572550.889884:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+353:2350:x1875453360622208:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 5256us (6732us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572550.889893:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 385 00000100:00100000:0.0:1788572550.906174:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.906230:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572550.906322:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360622720 00000100:00100000:3.0:1788572550.906332:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360622720 00000100:00100000:3.0:1788572550.906400:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 372 00000100:00100000:3.0:1788572550.906411:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+353:2350:x1875453360622720:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572550.906439:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572550.906463:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14aa expref: 353 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572550.906482:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6d:0x0].0x0 (ffff9627c5160e00) 00010000:00010000:3.0:1788572550.906487:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572550.906551:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+352:2350:x1875453360622720:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 142us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572550.906559:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 372 00000100:00100000:2.0:1788572550.906694:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360622592 00000100:00100000:2.0:1788572550.906699:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360622592 00002000:00100000:2.0:1788572550.906732:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x52:0x0] (0->4095) req@ffff9626c239b800 x1875453360622592/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/0 e 0 to 0 dl 1788572561 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.906742:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c239b800 with x1875453360622592 ext(0->4095) 00002000:00010000:2.0:1788572550.906746:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c239b800 00000100:00100000:2.0:1788572550.906772:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 354 00000100:00100000:2.0:1788572550.906779:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2349:x1875453360622592:12345-192.168.202.40@tcp:3:diff.0 00000100:00100000:2.0:1788572550.907054:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2349:x1875453360622592:12345-192.168.202.40@tcp:3:diff.0 Request processed in 275us (889us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788572550.907061:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 354 00002000:00100000:2.0:1788572550.907068:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x52:0x0] (0->4095) req@ffff9626c239b800 x1875453360622592/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:211/0 lens 488/4536 e 0 to 0 dl 1788572561 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572550.907076:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c239b800 with x1875453360622592 ext(0->4095) 00002000:00010000:2.0:1788572550.907080:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c239b800 00000100:00100000:1.0:1788572550.920482:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572550.920601:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360622976 00000100:00100000:0.0:1788572550.920610:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360622976 00000100:00100000:0.0:1788572550.920667:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1048 00000100:00100000:0.0:1788572550.920677:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+352:35867:x1875453360622976:12345-192.168.202.40@tcp:35:diff.0 00000100:00100000:2.0:1788572550.921860:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+353:35867:x1875453360622976:12345-192.168.202.40@tcp:35:diff.0 Request processed in 1170us (1375us total) trans 4294970954 rc 0/0 00000100:00100000:2.0:1788572550.921870:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1048 00010000:00010000:0.0:1788572551.290914:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572552.314291:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1788572552.506291:0:6873:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 900575283668ns, next at 901575280577ns, rc = 0 00000100:00100000:0.0:1788572552.506313:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788572552.506332:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382394112:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788572552.506400:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572552.506474:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394112 00000100:00100000:0.0:1788572552.506482:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394112 00000100:00100000:0.0:1788572552.506534:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1788572552.506542:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3304:x1875453382394112:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788572552.506575:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1788572552.506616:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3304:x1875453382394112:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572552.506623:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00080000:0.0:1788572552.506642:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572552.506686:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382394112:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788572553.019358:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3302:1875453382394240:0@lo:400:kworker.0 00000100:00100000:0.0:1788572553.019457:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572553.019517:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382394752:0@lo:400:kworker.0 00000100:00100000:0.0:1788572553.019544:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572553.019569:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788572553.019575:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7559180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382394368:0@lo:400:kworker.0 00000100:00100000:0.0:1788572553.019598:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572553.019617:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382394880:0@lo:400:kworker.0 00000100:00100000:0.0:1788572553.019640:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572553.019754:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394240 00000100:00100000:1.0:1788572553.019765:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394240 00000100:00100000:1.0:1788572553.019822:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1788572553.019834:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3302:x1875453382394240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572553.019975:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382394624:0@lo:400:kworker.0 00000100:00100000:0.0:1788572553.020049:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3304:1875453382394496:0@lo:400:kworker.0 00000100:00100000:2.0:1788572553.021860:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394752 00000100:00100000:2.0:1788572553.021874:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394752 00000100:00100000:2.0:1788572553.021930:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394368 00000100:00100000:2.0:1788572553.021933:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394368 00000100:00100000:2.0:1788572553.021956:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1788572553.021965:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382394752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.024788:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1788572553.024820:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382394368:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.024939:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382394368:12345-0@lo:400:kworker.0 Request processed in 130us (5339us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572553.024947:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1788572553.025040:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3302:x1875453382394240:12345-0@lo:400:kworker.0 Request processed in 5208us (5583us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572553.025049:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1788572553.025103:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394880 00000100:00100000:1.0:1788572553.025111:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394880 00000100:00100000:1.0:1788572553.025152:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:1.0:1788572553.025161:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+11:3302:x1875453382394880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.025196:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+11:3302:x1875453382394880:12345-0@lo:400:kworker.0 Request processed in 38us (5555us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572553.025202:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00080000:1.0:1788572553.025230:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788572553.025235:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572553.025269:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7559180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382394368:0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.025282:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382394880:0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.025292:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3302:1875453382394240:0@lo:400:kworker.0 00000100:00100000:3.0:1788572553.031783:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572553.035944:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394624 00000100:00100000:1.0:1788572553.035960:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394624 00000100:00100000:1.0:1788572553.036035:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 603 00000100:00100000:1.0:1788572553.036046:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382394624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.036152:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382394624:12345-0@lo:400:kworker.0 Request processed in 108us (4377us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572553.036160:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 603 00000100:00100000:1.0:1788572553.036347:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572553.036484:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382394624:0@lo:400:kworker.0 00000100:00100000:2.0:1788572553.037054:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382394752:12345-0@lo:400:kworker.0 Request processed in 15087us (17504us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572553.037067:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1788572553.037107:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382394752:0@lo:400:kworker.0 00000100:00100000:2.0:1788572553.037127:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382394496 00000100:00100000:2.0:1788572553.037132:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382394496 00000100:00080000:2.0:1788572553.037154:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572553.046822:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 638 00000100:00100000:1.0:1788572553.046969:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1875453382394496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572553.047096:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1875453382394496:12345-0@lo:400:kworker.0 Request processed in 260us (10749us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572553.047104:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 638 00000100:00100000:1.0:1788572553.047184:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3304:1875453382394496:0@lo:400:kworker.0 00000001:00080000:1.0:1788572553.105407:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967665 is committed 00000001:00080000:1.0:1788572553.105430:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967666 is committed 00000001:00080000:1.0:1788572553.105448:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967667 is committed 00010000:00010000:0.0:1788572553.342368:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1788572553.534731:0:7938:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 901603723997ns, next at 902603721120ns, rc = 0 00000100:00100000:1.0:1788572553.534800:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382395008:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788572553.534868:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572553.537295:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395008 00000100:00100000:0.0:1788572553.537311:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395008 00000100:00100000:0.0:1788572553.537366:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1788572553.537389:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382395008:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788572553.537430:0:6684:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788572553.537513:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382395008:12345-0@lo:13:osp-pre-1-0.0 Request processed in 137us (2643us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572553.537520:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1788572553.538738:0:3304:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967667 gen 2 00000004:00080000:1.0:1788572553.538750:0:3304:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b64380, transno 4294967660 00000004:00080000:1.0:1788572553.538759:0:3304:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b64000, transno 4294967661 00000004:00080000:1.0:1788572553.538762:0:3304:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b64e00, transno 4294967662 00000004:00080000:1.0:1788572553.538764:0:3304:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f5b64a80, transno 4294967663 00000004:00080000:1.0:1788572553.538769:0:3304:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967666 committed transno 4294967667 00000004:00080000:1.0:1788572553.538788:0:3304:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967667 committed transno 4294967667 00000100:00100000:1.0:1788572553.538813:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382395008:0@lo:13:osp-pre-1-0.0 00000004:00080000:2.0:1788572553.539448:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 4 done 00000001:00080000:2.0:1788572553.624117:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967669 is committed 00000001:00080000:2.0:1788572553.624150:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967668 is committed 00000004:00080000:2.0:1788572553.651294:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 46 committed 00000001:00080000:2.0:1788572553.651321:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970954 is committed 00000004:00080000:2.0:1788572553.651409:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 41 committed 00000001:00080000:2.0:1788572553.651479:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970832 is committed 00000020:00080000:2.0:1788572553.651484:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 144 batchid updated 00000001:00080000:2.0:1788572553.651564:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970837 is committed 00000020:00080000:2.0:1788572553.651571:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627e75b8d80 refcount 2 committed 0 result 0 batchid 145 00000020:00080000:2.0:1788572553.651576:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7ed00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627e75b8e40 00000020:00080000:2.0:1788572553.651579:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff962801f7e300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c10a8800 00000001:00080000:2.0:1788572553.651655:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970839 is committed 00000100:00100000:2.0:1788572553.653056:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1788572553.653079:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382395136:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572553.653145:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572553.653200:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382395264:0@lo:2:osp-syn-0-0.0 00000100:00100000:2.0:1788572553.653222:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572553.653239:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572553.653243:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382395392:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1788572553.653263:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572553.653289:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395136 00000100:00100000:2.0:1788572553.653295:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395136 00000100:00100000:2.0:1788572553.653334:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395264 00000100:00100000:2.0:1788572553.653337:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395264 00000100:00100000:2.0:1788572553.653349:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395392 00000100:00100000:2.0:1788572553.653351:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395392 00000100:00100000:2.0:1788572553.653367:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1400 00000100:00100000:2.0:1788572553.653389:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3304:x1875453382395136:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572553.653617:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572553.653620:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb mtime from disk: 1788572518 -> 1788572518 00002000:00010000:2.0:1788572553.653623:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb atime from disk: 1788572518 -> 1788572518 00002000:00010000:2.0:1788572553.653625:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb ctime from disk: 1788572526 -> 1788572526 00002000:00010000:2.0:1788572553.653627:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x59:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572553.653667:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3304:x1875453382395136:12345-0@lo:2:osp-syn-0-0.0 Request processed in 280us (521us total) trans 4294967670 rc 0/0 00000100:00100000:2.0:1788572553.653675:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1400 00000020:00080000:1.0:1788572553.653688:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 145 committed batchid 144 00000100:00100000:2.0:1788572553.653693:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1401 00000100:00100000:2.0:1788572553.653699:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382395264:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:2.0:1788572553.653770:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572553.653773:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb mtime from disk: 1788572517 -> 1788572517 00002000:00010000:2.0:1788572553.653775:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb atime from disk: 1788572517 -> 1788572517 00002000:00010000:2.0:1788572553.653777:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb ctime from disk: 1788572526 -> 1788572526 00002000:00010000:2.0:1788572553.653779:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5a:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572553.653808:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3302:x1875453382395264:12345-0@lo:2:osp-syn-0-0.0 Request processed in 109us (584us total) trans 4294967671 rc 0/0 00000100:00100000:2.0:1788572553.653814:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1401 00000100:00100000:2.0:1788572553.653828:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1402 00000100:00100000:2.0:1788572553.653834:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1875453382395392:12345-0@lo:2:osp-syn-1-0.0 00000020:00080000:1.0:1788572553.653991:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 144 00002000:00010000:2.0:1788572553.655414:0:6679:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb size from disk: 3 -> 3 00002000:00010000:2.0:1788572553.655422:0:6679:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb mtime from disk: 1788572517 -> 1788572517 00002000:00010000:2.0:1788572553.655425:0:6679:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb atime from disk: 1788572517 -> 1788572517 00002000:00010000:2.0:1788572553.655427:0:6679:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb ctime from disk: 1788572526 -> 1788572526 00002000:00010000:2.0:1788572553.655429:0:6679:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x59:0x0] updating lvb blocks from disk: 11 -> 11 00000100:00100000:2.0:1788572553.655484:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382395392:12345-0@lo:2:osp-syn-1-0.0 Request processed in 1648us (2217us total) trans 4294967668 rc 0/0 00000100:00100000:2.0:1788572553.655492:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1402 00000100:00100000:2.0:1788572553.655542:0:3304:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967669 gen 2 00000004:00080000:2.0:1788572553.655548:0:3304:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967669 committed transno 4294967669 00000100:00100000:2.0:1788572553.655569:0:3304:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627ff0a8a80 x1875453382395136/t4294967670(4294967670) o2->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 440/432 e 0 to 0 dl 1788572569 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1788572553.655581:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ff0a8a80/2, rc 0, transno 4294967670 00000100:00100000:2.0:1788572553.655585:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382395136:0@lo:2:osp-syn-0-0.0 00000004:00080000:2.0:1788572553.655610:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ff0a8700/2, rc 0, transno 4294967668 00000100:00100000:2.0:1788572553.655614:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382395392:0@lo:2:osp-syn-1-0.0 00000004:00080000:2.0:1788572553.655621:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627ff0a9500/2, rc 0, transno 4294967671 00000100:00100000:2.0:1788572553.655624:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382395264:0@lo:2:osp-syn-0-0.0 00010000:00010000:3.0:1788572554.365567:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572555.393156:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572555.901417:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901460:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901488:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901505:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901525:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901541:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901565:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572555.901592:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623360 00000100:00100000:0.0:1788572555.901602:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623360 00000100:00100000:0.0:1788572555.901660:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623872 00000100:00100000:0.0:1788572555.901663:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623872 00000100:00100000:0.0:1788572555.901679:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623488 00000100:00100000:0.0:1788572555.901681:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623488 00000100:00100000:0.0:1788572555.901708:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623744 00000100:00100000:0.0:1788572555.901711:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623744 00000100:00100000:0.0:1788572555.901726:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 604 00000100:00100000:0.0:1788572555.901737:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2347:x1875453360623360:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572555.901807:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2347:x1875453360623360:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 71us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572555.901814:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 604 00000100:00100000:0.0:1788572555.901832:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 605 00000100:00100000:0.0:1788572555.901841:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2347:x1875453360623872:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572555.901865:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2347:x1875453360623872:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 27us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572555.901871:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 605 00000100:00100000:0.0:1788572555.901882:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 606 00000100:00100000:0.0:1788572555.901890:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2347:x1875453360623488:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572555.901912:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2347:x1875453360623488:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 24us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572555.901918:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 606 00000100:00100000:0.0:1788572555.901929:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 607 00000100:00100000:0.0:1788572555.901937:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2347:x1875453360623744:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572555.901958:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2347:x1875453360623744:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 24us (393us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572555.901965:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 607 00000100:00100000:2.0:1788572555.902074:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623104 00000100:00100000:2.0:1788572555.902080:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623104 00000100:00100000:2.0:1788572555.902109:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 683 00000100:00100000:2.0:1788572555.902116:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0effc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2347:x1875453360623104:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572555.902144:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0effc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2347:x1875453360623104:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 29us (738us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572555.902151:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 683 00000100:00080000:2.0:1788572555.902178:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788572555.902182:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572555.902433:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623616 00000100:00100000:1.0:1788572555.902438:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623616 00000100:00100000:1.0:1788572555.902480:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360623232 00000100:00100000:1.0:1788572555.902483:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360623232 00000100:00100000:1.0:1788572555.902504:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 639 00000100:00100000:1.0:1788572555.902512:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:2347:x1875453360623616:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572555.902543:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:2347:x1875453360623616:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 32us (1055us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572555.902549:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 639 00000100:00100000:1.0:1788572555.902561:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 640 00000100:00100000:1.0:1788572555.902567:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:2347:x1875453360623232:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572555.902613:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:2347:x1875453360623232:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 46us (1087us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572555.902620:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 640 00000100:00080000:2.0:1788572555.902864:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1788572556.410831:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572557.434211:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572557.630179:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788572557.630202:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382395520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788572557.630285:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572557.630343:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395520 00000100:00100000:0.0:1788572557.630349:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395520 00000100:00100000:0.0:1788572557.630403:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:0.0:1788572557.630412:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382395520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788572557.630460:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1788572557.630507:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382395520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572557.630514:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00080000:0.0:1788572557.630538:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572557.630574:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382395520:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788572558.142145:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382396032:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142264:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572558.142334:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1788572558.142340:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3304:1875453382395648:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142374:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572558.142399:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382396160:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142426:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572558.142452:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382395776:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142476:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572558.142490:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382396288:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142511:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572558.142530:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788572558.142535:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3302:1875453382395904:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142556:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572558.142593:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382396032 00000100:00100000:3.0:1788572558.142601:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382396032 00000100:00100000:3.0:1788572558.142656:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382396288 00000100:00100000:3.0:1788572558.142659:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382396288 00000100:00100000:3.0:1788572558.142696:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:3.0:1788572558.142705:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3304:x1875453382396032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142765:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3304:x1875453382396032:12345-0@lo:400:kworker.0 Request processed in 60us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572558.142773:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:3.0:1788572558.142791:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1788572558.142799:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382396288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142830:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382396288:12345-0@lo:400:kworker.0 Request processed in 32us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572558.142837:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1788572558.142861:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395648 00000100:00100000:3.0:1788572558.142864:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395648 00000100:00100000:3.0:1788572558.142890:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:3.0:1788572558.142897:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efe850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3304:x1875453382395648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.142925:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efe850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3304:x1875453382395648:12345-0@lo:400:kworker.0 Request processed in 29us (550us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572558.142930:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:3.0:1788572558.142966:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382396160 00000100:00100000:3.0:1788572558.142970:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382396160 00000100:00100000:3.0:1788572558.142995:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395776 00000100:00100000:3.0:1788572558.142998:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395776 00000100:00100000:3.0:1788572558.143044:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382395904 00000100:00100000:3.0:1788572558.143047:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382395904 00000100:00100000:3.0:1788572558.143071:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:3.0:1788572558.143078:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382396160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143111:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382396160:12345-0@lo:400:kworker.0 Request processed in 34us (684us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572558.143117:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:3.0:1788572558.143130:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:3.0:1788572558.143136:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382395776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143163:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382395776:12345-0@lo:400:kworker.0 Request processed in 27us (686us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572558.143169:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:3.0:1788572558.143181:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:3.0:1788572558.143188:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1875453382395904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143232:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1875453382395904:12345-0@lo:400:kworker.0 Request processed in 44us (674us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572558.143238:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00080000:3.0:1788572558.143265:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788572558.143270:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788572558.143304:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3304:1875453382395648:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143320:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382396160:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143332:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382396032:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143373:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3302:1875453382395904:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143383:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382395776:0@lo:400:kworker.0 00000100:00100000:3.0:1788572558.143393:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382396288:0@lo:400:kworker.0 00000100:00100000:0.0:1788572558.174335:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572558.174507:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624000 00000100:00100000:1.0:1788572558.174521:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624000 00000100:00100000:1.0:1788572558.174602:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4828 00000100:00100000:1.0:1788572558.174613:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:35692:x1875453360624000:12345-192.168.202.40@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1788572558.174739:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:35692:x1875453360624000:12345-192.168.202.40@tcp:60:lhsmtool_posix.0 Request processed in 127us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572558.174747:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4828 00000100:00080000:3.0:1788572558.176474:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1788572558.183232:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967668 is committed 00010000:00010000:2.0:1788572558.458650:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572558.650296:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7b9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382396416:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788572558.650418:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572558.650539:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382396416 00000100:00100000:1.0:1788572558.650549:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382396416 00000100:00100000:1.0:1788572558.650620:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1788572558.650632:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382396416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788572558.650855:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788572558.650922:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382396416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 293us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572558.650932:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1788572558.651059:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967668 gen 2 00000004:00080000:1.0:1788572558.651067:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ff0a8700, transno 4294967668 00000100:00100000:1.0:1788572558.651096:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7b9e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382396416:0@lo:13:osp-pre-1-0.0 00000020:00080000:2.0:1788572558.725140:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 145 batchid updated 00000020:00080000:1.0:1788572558.725202:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 145 00000020:00080000:1.0:1788572558.725218:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 145 00000001:00080000:0.0:1788572558.742786:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967671 is committed 00010000:00010000:0.0:1788572559.482286:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1788572559.615414:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572559.615434:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572559.656299:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 166 count 1 00000040:00080000:3.0:1788572559.656324:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:02000400:0.0:1788572560.465030:0:23496:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 21:42:38 (1788572558) 00010000:00010000:3.0:1788572560.506375:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572560.846956:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572560.847144:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624128 00000100:00100000:3.0:1788572560.847156:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624128 00000100:00100000:3.0:1788572560.847246:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4829 00000100:00100000:3.0:1788572560.847257:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:36573:x1875453360624128:12345-192.168.202.40@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1788572560.847372:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:36573:x1875453360624128:12345-192.168.202.40@tcp:59:lhsmtool_posix.0 Request processed in 116us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572560.847379:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4829 00000100:00080000:0.0:1788572560.847777:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572560.899072:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572560.899389:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624256 00000100:00100000:2.0:1788572560.899399:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624256 00000100:00100000:2.0:1788572560.899478:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4830 00000100:00100000:2.0:1788572560.899488:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:36574:x1875453360624256:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572560.899509:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572560.899530:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1656e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14b1 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.899615:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.899631:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.899643:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.899659:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.899666:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.899673:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788572560.899871:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1788572560.900157:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900246:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900253:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900259:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572560.900263:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900270:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900283:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:2.0:1788572560.900286:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572560.900290:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900296:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb1656ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900306:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900312:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900317:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900321:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572560.900324:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900328:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900333:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd097bed6 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572560.900335:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572560.900338:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900342:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1656f3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900372:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1656e5 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14b1 expref: 239 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900381:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:2.0:1788572560.900383:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572560.900386:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1656e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14b1 expref: 239 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.900399:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496800, rc 0) 00000100:00100000:2.0:1788572560.900464:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:36574:x1875453360624256:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 977us (1403us total) trans 0 rc 301/301 00000100:00100000:2.0:1788572560.900471:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4830 00000100:00100000:1.0:1788572560.907670:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572560.908257:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624512 00000100:00100000:3.0:1788572560.908269:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624512 00000100:00100000:3.0:1788572560.908450:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4831 00000100:00100000:3.0:1788572560.908463:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:36574:x1875453360624512:12345-192.168.202.40@tcp:36:lfs.0 00010000:00010000:3.0:1788572560.908526:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572560.908554:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c478ec00/0x85ace67feb15b93b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6ed51e expref: 238 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.908573:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:3.0:1788572560.908578:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572560.909377:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.909395:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 55 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.909404:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.909437:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 55 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.909451:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 55 pid: 22536 timeout: 1008 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572560.909468:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f90000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382396544:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572560.909574:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572560.916541:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572560.916762:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624640 00000100:00100000:0.0:1788572560.916773:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624640 00010000:00010000:0.0:1788572560.916835:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 56 pid: 22536 timeout: 1008 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572560.916887:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1125 00000100:00100000:0.0:1788572560.916900:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:2349:x1875453360624640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572560.916919:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572560.916926:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 56 pid: 22536 timeout: 1008 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.916950:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 56 pid: 22536 timeout: 1008 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.916977:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:0.0:1788572560.916984:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.916994:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.917022:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.917046:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572560.917119:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:2349:x1875453360624640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 222us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572560.917127:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1125 00010000:00010000:1.0:1788572560.919280:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7a97c00/0x85ace67feb15d95c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6eddb4 expref: 55 pid: 22536 timeout: 1008 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572560.919303:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f90000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382396544:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572560.919330:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572560.919340:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572560.919345:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572560.919376:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572560.919387:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572560.919397:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572560.923422:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00010000:00010000:0.0:1788572560.923447:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.923459:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.923468:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.923474:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572560.923478:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.923487:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.923509:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:0.0:1788572560.923513:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572560.923518:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.923524:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1656fa lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572560.923549:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004cd7013b reply state 00000000749f5f4b transno 4294970955 ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572560.923571:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572560.923741:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:36574:x1875453360624512:12345-192.168.202.40@tcp:36:lfs.0 Request processed in 15277us (16077us total) trans 4294970955 rc 0/0 00000100:00100000:0.0:1788572560.923751:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4831 00010000:00010000:2.0:1788572560.930171:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.930191:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.930199:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.930206:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572560.930211:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.930220:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.930236:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd097bed6 (ffff9627ff5ab500) 00010000:00010000:2.0:1788572560.930239:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572560.930245:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572560.930251:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165701 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572560.967055:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572560.967262:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624768 00000100:00100000:3.0:1788572560.967272:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624768 00000100:00100000:3.0:1788572560.967337:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4832 00000100:00100000:3.0:1788572560.967349:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+55:36578:x1875453360624768:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:3.0:1788572560.967373:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572560.967397:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165708 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14b8 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.967465:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16570f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.967479:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16570f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.967493:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16570f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572560.967530:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788572560.967534:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572560.967554:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16570f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.967739:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165708 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14b8 expref: 57 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.967772:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16570f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14b8 expref: 56 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572560.967782:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:3.0:1788572560.967785:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572560.967790:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5800, rc 0) 00000100:00100000:3.0:1788572560.967922:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:36578:x1875453360624768:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 575us (877us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572560.967942:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4832 00000100:00100000:3.0:1788572560.979642:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572560.979681:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572560.979899:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360624896 00000100:00100000:1.0:1788572560.979913:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360624896 00000100:00100000:1.0:1788572560.979985:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625152 00000100:00100000:1.0:1788572560.979990:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625152 00000100:00100000:1.0:1788572560.980048:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4833 00000100:00100000:1.0:1788572560.980060:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2348:x1875453360624896:12345-192.168.202.40@tcp:41:mkdir.0 00000100:00100000:0.0:1788572560.980114:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4834 00000004:00000024:1.0:1788572560.980118:0:5838:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259419136 avail 2257321984 00000100:00100000:0.0:1788572560.980140:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36578:x1875453360625152:12345-192.168.202.40@tcp:36:mkdir.0 00010000:00010000:0.0:1788572560.980188:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1788572560.980194:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2348:x1875453360624896:12345-192.168.202.40@tcp:41:mkdir.0 Request processed in 137us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572560.980201:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4833 00010000:00010000:0.0:1788572560.980216:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb16570f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14b8 expref: 57 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.980235:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:0.0:1788572560.980245:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572560.980378:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+55:36578:x1875453360625152:12345-192.168.202.40@tcp:36:mkdir.0 Request processed in 254us (695us total) trans 0 rc -17/-17 00000100:00100000:0.0:1788572560.980385:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4834 00000100:00100000:3.0:1788572560.987583:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572560.987753:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625280 00000100:00100000:0.0:1788572560.987763:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625280 00000100:00100000:0.0:1788572560.987845:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4835 00000100:00100000:0.0:1788572560.987975:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+55:36578:x1875453360625280:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:0.0:1788572560.987997:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572560.988172:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165716 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14bf expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.988251:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.988265:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.988277:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572560.988315:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788572560.988318:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572560.988337:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.988360:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165716 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 57 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.988380:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 56 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572560.988389:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:0.0:1788572560.988391:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572560.988395:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aea00, rc 0) 00000100:00100000:0.0:1788572560.988471:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:36578:x1875453360625280:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 618us (897us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572560.988479:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4835 00000100:00100000:1.0:1788572560.995174:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.001957:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625408 00000100:00100000:2.0:1788572561.001974:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625408 00000100:00100000:2.0:1788572561.004592:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4836 00000100:00100000:2.0:1788572561.004632:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+56:36578:x1875453360625408:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:2.0:1788572561.004657:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.004682:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165724 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14c6 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004761:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004776:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004799:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004815:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004822:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004828:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004926:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165739 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004934:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165739 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.004947:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165739 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572561.004980:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00000004:00200000:2.0:1788572561.004984:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572561.005214:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165739 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005228:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005235:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005242:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005248:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.005252:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005261:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005273:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:2.0:1788572561.005276:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.005280:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005286:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb16572b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005302:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005307:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005312:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005316:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.005319:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005324:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005330:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd097bed6 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572561.005332:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.005335:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005341:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb165732 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005373:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165724 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14c6 expref: 58 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005389:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165739 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14c6 expref: 57 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.005400:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19900) 00010000:00010000:2.0:1788572561.005402:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572561.005404:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496400, rc 0) 00000100:00100000:2.0:1788572561.005478:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36578:x1875453360625408:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 873us (10315us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.005487:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4836 00000100:00100000:1.0:1788572561.007281:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572561.007328:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572561.007356:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572561.007378:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572561.007406:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572561.007431:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625664 00000100:00100000:1.0:1788572561.007441:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625664 00000100:00100000:1.0:1788572561.007490:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625920 00000100:00100000:1.0:1788572561.007494:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625920 00000100:00100000:1.0:1788572561.007525:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625792 00000100:00100000:1.0:1788572561.007527:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625792 00000100:00100000:1.0:1788572561.007538:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626048 00000100:00100000:1.0:1788572561.007540:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626048 00000100:00100000:1.0:1788572561.007558:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 610 00000100:00100000:1.0:1788572561.007566:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2348:x1875453360625664:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572561.007641:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2348:x1875453360625664:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 75us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572561.007648:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 610 00000100:00100000:1.0:1788572561.007670:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 611 00000100:00100000:1.0:1788572561.007676:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360625920:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572561.007704:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360625920:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 28us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572561.007710:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 611 00000100:00100000:1.0:1788572561.007722:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 612 00000100:00100000:1.0:1788572561.007728:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2348:x1875453360625792:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572561.007752:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2348:x1875453360625792:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 24us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572561.007757:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 612 00000100:00100000:1.0:1788572561.007769:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 613 00000100:00100000:1.0:1788572561.007775:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360626048:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572561.007803:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360626048:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 28us (396us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572561.007809:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 613 00000100:00100000:1.0:1788572561.007828:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360625536 00000100:00100000:1.0:1788572561.007831:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360625536 00000100:00100000:1.0:1788572561.007854:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 685 00000100:00100000:1.0:1788572561.007860:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0eff850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360625536:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572561.007886:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0eff850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360625536:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 26us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572561.007893:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 685 00000100:00080000:1.0:1788572561.007921:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788572561.007925:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572561.017987:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.018504:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626176 00000100:00100000:2.0:1788572561.018517:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626176 00000100:00100000:2.0:1788572561.018588:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 386 00000100:00100000:2.0:1788572561.018601:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2349:x1875453360626176:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572561.018640:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.018662:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb165740 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14cd expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.018728:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.018743:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.018755:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.018770:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.018777:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.018785:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019316:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019325:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019333:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019339:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572561.019343:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019350:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019364:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572561.019366:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572561.019372:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019378:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb165747 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019395:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019402:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019407:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019412:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572561.019415:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019421:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019429:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x15ea4583 (ffff9627ffc19d00) 00010000:00010000:3.0:1788572561.019431:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572561.019434:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019439:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb16574e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x15ea4583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019472:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb165740 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14cd expref: 58 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019482:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572561.019484:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572561.019489:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb165740 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14cd expref: 58 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.019498:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497800, rc -2) 00000100:00100000:3.0:1788572561.019590:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2349:x1875453360626176:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 991us (1613us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572561.019599:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 386 00000100:00100000:1.0:1788572561.040827:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.041117:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626304 00000100:00100000:2.0:1788572561.041127:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626304 00000100:00100000:2.0:1788572561.041195:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4837 00000100:00100000:2.0:1788572561.041206:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36579:x1875453360626304:12345-192.168.202.40@tcp:101:rm.0 00010000:00010000:2.0:1788572561.041230:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.041254:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165755 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14d4 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041354:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 3/1,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041370:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041383:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041398:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041404:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041410:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041482:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041490:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041497:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041503:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.041508:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041515:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041528:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19900) 00010000:00010000:2.0:1788572561.041531:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.041537:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041543:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16575c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041554:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041560:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041566:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041571:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7497200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.041574:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041580:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041602:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572561.041606:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.041609:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041615:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165763 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041652:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165755 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14d4 expref: 58 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041663:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19900) 00010000:00010000:2.0:1788572561.041666:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.041670:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496c00/0x85ace67feb165755 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14d4 expref: 58 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.041678:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496c00, rc 0) 00000100:00100000:2.0:1788572561.041751:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36579:x1875453360626304:12345-192.168.202.40@tcp:101:rm.0 Request processed in 546us (934us total) trans 0 rc 301/301 00000100:00100000:2.0:1788572561.041758:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4837 00000100:00100000:1.0:1788572561.063274:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.063400:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626560 00000100:00100000:0.0:1788572561.063412:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626560 00000100:00100000:0.0:1788572561.063499:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4838 00000100:00100000:0.0:1788572561.063513:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36580:x1875453360626560:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572561.063535:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572561.063539:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572561.063576:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb165739 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14c6 expref: 57 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063611:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19900) 00010000:00010000:0.0:1788572561.063624:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572561.063643:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb16576a lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14db expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063809:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063825:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063838:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063852:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063859:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.063866:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572561.064686:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x5a:0x0], index 1: 0 00010000:00010000:0.0:1788572561.064892:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb16577f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064906:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb16577f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064916:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb16577f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064951:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064958:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064966:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064973:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572561.064977:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064985:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.064995:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:0.0:1788572561.064998:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572561.065037:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.065044:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165771 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572561.065055:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9f629c5 reply state 0000000053e95d53 transno 4294970956 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572561.065064:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788572561.065109:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb16576a lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14db expref: 59 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.065129:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb16577f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14db expref: 58 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.065137:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:0.0:1788572561.065139:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572561.065141:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af800, rc 0) 00000100:00100000:0.0:1788572561.065222:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36580:x1875453360626560:12345-192.168.202.40@tcp:101:dd.0 Request processed in 1713us (1959us total) trans 4294970956 rc 0/0 00000100:00100000:0.0:1788572561.065232:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4838 00010000:00010000:3.0:1788572561.069820:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.069847:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.069855:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.069861:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572561.069866:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.069874:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.069891:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627c5160900) 00010000:00010000:3.0:1788572561.069895:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572561.069900:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.069907:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165778 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572561.129262:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.129408:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626688 00000100:00100000:2.0:1788572561.129420:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626688 00000100:00100000:2.0:1788572561.129486:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1403 00000100:00100000:2.0:1788572561.129495:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:36580:x1875453360626688:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:2.0:1788572561.129513:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.129536:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14e2 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572561.129565:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1788572561.129588:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=909s v=(0 0) 00010000:00010000:2.0:1788572561.129593:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572561.129604:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 10 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.129622:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 10 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.129636:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7496c00, rc 0) 00000100:00100000:2.0:1788572561.129703:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:36580:x1875453360626688:12345-192.168.202.40@tcp:101:dd.0 Request processed in 209us (452us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.129709:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1403 00000100:00100000:1.0:1788572561.171739:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.171878:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626816 00000100:00100000:2.0:1788572561.171889:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626816 00002000:00100000:2.0:1788572561.171942:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5a:0x0] (0->1048575) req@ffff9626c2c4ad80 x1875453360626816/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:222/0 lens 488/0 e 0 to 0 dl 1788572572 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572561.171952:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2c4ad80 with x1875453360626816 ext(0->1048575) 00002000:00010000:2.0:1788572561.171957:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2c4ad80 00000100:00100000:2.0:1788572561.171996:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 355 00000100:00100000:2.0:1788572561.172034:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360626816:12345-192.168.202.40@tcp:4:dd.0 00000100:00100000:3.0:1788572561.202849:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360626816:12345-192.168.202.40@tcp:4:dd.0 Request processed in 30832us (31108us total) trans 4294967669 rc 0/0 00000100:00100000:3.0:1788572561.202865:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 355 00002000:00100000:3.0:1788572561.202880:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5a:0x0] (0->1048575) req@ffff9626c2c4ad80 x1875453360626816/t4294967669(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:222/0 lens 488/448 e 0 to 0 dl 1788572572 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572561.202891:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2c4ad80 with x1875453360626816 ext(0->1048575) 00002000:00010000:3.0:1788572561.202899:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c2c4ad80 00000100:00100000:1.0:1788572561.212253:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.212357:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360626944 00000100:00100000:0.0:1788572561.212367:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360626944 00000100:00100000:0.0:1788572561.212455:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4839 00000100:00100000:0.0:1788572561.212468:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:36580:x1875453360626944:12345-192.168.202.40@tcp:44:dd.0 00000001:00080000:3.0:1788572561.260661:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970956 is committed 00000001:00080000:3.0:1788572561.260679:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970955 is committed 00000100:00100000:0.0:1788572561.261198:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36580:x1875453360626944:12345-192.168.202.40@tcp:44:dd.0 Request processed in 48727us (48950us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572561.261210:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4839 00000100:00100000:2.0:1788572561.272939:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572561.275274:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360627072 00000100:00100000:3.0:1788572561.275289:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360627072 00000100:00100000:3.0:1788572561.275361:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1404 00000100:00100000:3.0:1788572561.275371:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360627072:12345-192.168.202.40@tcp:16:dd.0 00000001:00080000:0.0:1788572561.342312:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967669 is committed 00000100:00100000:2.0:1788572561.352730:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360627072:12345-192.168.202.40@tcp:16:dd.0 Request processed in 77344us (79786us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.352749:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1404 00000100:00100000:1.0:1788572561.357093:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572561.357320:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360627328 00000100:00100000:3.0:1788572561.357335:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360627328 00000100:00100000:3.0:1788572561.357415:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1049 00000100:00100000:3.0:1788572561.357429:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36580:x1875453360627328:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:3.0:1788572561.357939:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:36580:x1875453360627328:12345-192.168.202.40@tcp:35:dd.0 Request processed in 512us (868us total) trans 4294970957 rc 0/0 00000100:00100000:3.0:1788572561.357950:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1049 00000100:00100000:1.0:1788572561.403239:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.403361:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360627456 00000100:00100000:2.0:1788572561.403374:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360627456 00000100:00100000:2.0:1788572561.404211:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4840 00000100:00100000:2.0:1788572561.404229:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:36581:x1875453360627456:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572561.404264:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.404292:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165794 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14e9 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404394:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404414:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404428:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404448:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404455:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404462:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404848:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1657a9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404858:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1657a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404865:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1657a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572561.404899:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00000004:00200000:2.0:1788572561.404902:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572561.404927:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1657a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404940:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404948:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404955:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404961:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.404966:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.404974:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405000:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:2.0:1788572561.405055:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.405061:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405077:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb16579b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405091:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405097:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405102:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405107:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.405110:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405117:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405124:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd097bed6 (ffff9627ffc19a00) 00010000:00010000:2.0:1788572561.405126:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.405129:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405134:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657a2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405168:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb165794 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e9 expref: 60 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405187:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497800/0x85ace67feb1657a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e9 expref: 59 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.405195:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:2.0:1788572561.405198:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1788572561.405200:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497800, rc 0) 00000100:00100000:2.0:1788572561.405292:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36581:x1875453360627456:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1069us (2070us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.405302:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4840 00000100:00100000:1.0:1788572561.482847:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.486032:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360627584 00000100:00100000:0.0:1788572561.486057:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360627584 00000100:00100000:0.0:1788572561.486186:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4841 00000100:00100000:0.0:1788572561.486201:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+237:36584:x1875453360627584:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572561.486234:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572561.486262:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1657b0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14f0 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.486363:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1657b7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.486382:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1657b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.486396:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1657b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572561.486451:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788572561.486456:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572561.486483:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1657b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.486513:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1657b0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14f0 expref: 239 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.486535:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1657b7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14f0 expref: 238 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.486545:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:0.0:1788572561.486548:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572561.486552:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae400, rc 0) 00000100:00100000:0.0:1788572561.486643:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:36584:x1875453360627584:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 446us (3809us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572561.486651:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4841 00000100:00100000:1.0:1788572561.497391:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.498178:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360627712 00000100:00100000:2.0:1788572561.498203:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360627712 00000100:00100000:2.0:1788572561.498328:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4842 00000100:00100000:2.0:1788572561.498344:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+238:36584:x1875453360627712:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572561.498385:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.498419:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb1657be lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14f7 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498507:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498527:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498541:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498560:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498568:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498575:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498945:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1657d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498955:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1657d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.498962:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1657d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572561.498996:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00000004:00200000:2.0:1788572561.499000:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572561.499102:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1657d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499118:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499124:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499132:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499139:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.499145:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499156:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499171:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:2.0:1788572561.499175:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.499183:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499190:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496400/0x85ace67feb1657c5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499207:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499213:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499219:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499224:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.499227:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499233:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499239:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd097bed6 (ffff9627ffc19a00) 00010000:00010000:2.0:1788572561.499242:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.499245:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499255:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496e00/0x85ace67feb1657cc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd097bed6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499302:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497200/0x85ace67feb1657be lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14f7 expref: 240 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499316:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497a00/0x85ace67feb1657d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14f7 expref: 239 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.499323:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:2.0:1788572561.499326:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1788572561.499328:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7497a00, rc 0) 00000100:00100000:2.0:1788572561.499413:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:36584:x1875453360627712:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 1072us (2033us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.499422:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4842 00000100:00100000:1.0:1788572561.504976:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.506677:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360627968 00000100:00100000:2.0:1788572561.506699:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360627968 00000100:00100000:2.0:1788572561.506815:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4843 00000100:00100000:2.0:1788572561.506829:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:36584:x1875453360627968:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572561.506858:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.506889:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb1657da lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f14fe expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507002:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507040:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507055:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507084:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507091:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507098:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507543:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507554:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507562:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572561.507573:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa7:0x0]: returning size 1048576 00010000:00010000:2.0:1788572561.507696:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507705:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507712:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507718:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.507724:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507732:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507746:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:2.0:1788572561.507750:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.507755:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507761:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496000/0x85ace67feb1657e1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507776:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507782:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507787:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507791:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e7496800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.507794:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507800:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507806:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627ffc19800) 00010000:00010000:2.0:1788572561.507809:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.507812:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507817:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7496800/0x85ace67feb1657e8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507872:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e7497000/0x85ace67feb1657da lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 242 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507888:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 241 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.507895:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:2.0:1788572561.507898:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572561.507900:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715600, rc 0) 00000100:00100000:2.0:1788572561.508041:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:36584:x1875453360627968:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 1176us (3036us total) trans 4294970958 rc 0/0 00000100:00100000:2.0:1788572561.508052:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4843 00000100:00100000:1.0:1788572561.513400:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572561.513586:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628096 00000100:00100000:3.0:1788572561.513604:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628096 00000100:00100000:3.0:1788572561.513751:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1405 00000100:00100000:3.0:1788572561.513765:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2350:x1875453360628096:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572561.513792:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572561.513813:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ba018e50/0x85ace67feb1657fd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1505 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572561.513893:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382396672:192.168.202.40@tcp:106: 00000100:00100000:3.0:1788572561.514054:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572561.520299:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572561.522088:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.522279:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628224 00000100:00100000:2.0:1788572561.522308:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628224 00000100:00100000:2.0:1788572561.522381:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628352 00000100:00100000:2.0:1788572561.522385:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628352 00000100:00100000:2.0:1788572561.522408:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 387 00000100:00100000:2.0:1788572561.522420:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2349:x1875453360628224:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572561.522513:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.522541:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165804 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f150c expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.522622:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572561.522634:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 388 00010000:00010000:2.0:1788572561.522655:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572561.522658:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360628352:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572561.522689:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.522711:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.522715:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.522719:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.522728:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.522983:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165819 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1513 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.522996:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523044:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523051:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523058:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.523075:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523084:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523099:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572561.523103:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572561.523105:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523120:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.523125:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523134:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb16580b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523152:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523158:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523163:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523168:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572561.523171:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523176:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523184:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0xe53fd9b7 (ffff9627d20cb300) 00010000:00010000:2.0:1788572561.523186:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572561.523189:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.523194:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165812 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0xe53fd9b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.523672:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.523696:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.523704:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.523711:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.524294:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165804 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f150c expref: 62 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.524324:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:2.0:1788572561.524327:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572561.524332:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715200/0x85ace67feb165804 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f150c expref: 62 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.524348:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c3715200, rc -2) 00000100:00100000:2.0:1788572561.524465:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2349:x1875453360628224:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2045us (4172us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572561.524478:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 387 00010000:00010000:1.0:1788572561.524595:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524606:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524613:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524618:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff962802050400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572561.524622:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524634:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524647:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:1.0:1788572561.524650:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572561.524656:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524663:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802050400/0x85ace67feb165820 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524677:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524682:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524687:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524691:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c246c000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572561.524694:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524699:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524706:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x76f43da (ffff9627ff5ab000) 00010000:00010000:1.0:1788572561.524709:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572561.524712:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524721:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c246c000/0x85ace67feb165827 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x76f43da bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524771:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165819 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1513 expref: 60 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524782:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:1.0:1788572561.524785:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572561.524791:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff962802051200/0x85ace67feb165819 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1513 expref: 60 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.524805:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff962802051200, rc -2) 00002000:00010000:3.0:1788572561.524821:0:6681:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x5a:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:3.0:1788572561.524828:0:6681:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x5a:0x0] updating lvb mtime: 0 -> 1788572560 00002000:00010000:3.0:1788572561.524831:0:6681:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x5a:0x0] updating lvb atime: 0 -> 1788572560 00002000:00010000:3.0:1788572561.524833:0:6681:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x5a:0x0] updating lvb ctime: 0 -> 1788572560 00002000:00010000:3.0:1788572561.524835:0:6681:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x5a:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:3.0:1788572561.524862:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382396672:192.168.202.40@tcp:106: 00000100:00100000:1.0:1788572561.524880:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:2350:x1875453360628352:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2227us (2794us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572561.524893:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 388 00010000:00010000:3.0:1788572561.524917:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ba018e50/0x85ace67feb1657fd lrc: 1/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1505 expref: 83 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.524939:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ba018e50/0x85ace67feb1657fd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1505 expref: 83 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.524950:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4200, rc 0) 00000100:00100000:3.0:1788572561.524992:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2350:x1875453360628096:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 11231us (11601us total) trans 0 rc 301/301 00000100:00100000:3.0:1788572561.525022:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1405 00010000:00010000:3.0:1788572561.530718:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572561.532445:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.533174:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628480 00000100:00100000:2.0:1788572561.533191:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628480 00000100:00100000:2.0:1788572561.533267:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1406 00000100:00100000:2.0:1788572561.533281:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:36584:x1875453360628480:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572561.533318:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.533343:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f151a expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.533385:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 9 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.533396:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x5a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572561.533402:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572561.533408:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 4/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 83 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.533459:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 9 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.533474:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 9 pid: 6679 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572561.533491:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382396800:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572561.533631:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e74950c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572561.539554:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382396800:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572561.539615:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 3/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 83 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.539643:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8800, rc 0) 00000100:00100000:2.0:1788572561.539719:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:36584:x1875453360628480:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 6441us (7282us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.539730:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1406 00000100:00100000:1.0:1788572561.546705:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572561.546940:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628608 00000100:00100000:3.0:1788572561.546952:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628608 00010000:00010000:3.0:1788572561.546997:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 10 pid: 6679 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572561.547088:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1126 00000100:00100000:3.0:1788572561.547099:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360628608:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572561.547114:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572561.547148:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 10 pid: 6679 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.547164:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 10 pid: 6679 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.547188:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007787d154/0x85ace67feb16578d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14e2 expref: 10 pid: 6679 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.547206:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572561.547211:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572561.547219:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 82 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.547228:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 82 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.547270:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 82 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572561.547285:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382396928:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572561.547396:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1788572561.552455:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382396928:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572561.552501:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572561.552575:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360628608:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 5477us (5880us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572561.552586:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1126 00000100:00100000:1.0:1788572561.556513:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.556595:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628736 00000100:00100000:2.0:1788572561.556615:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628736 00002000:00100000:2.0:1788572561.556693:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5a:0x0] (0->32767) req@ffff9627f5b64e00 x1875453360628736/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:222/0 lens 488/0 e 0 to 0 dl 1788572572 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572561.556706:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5b64e00 with x1875453360628736 ext(0->32767) 00002000:00010000:2.0:1788572561.556713:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627f5b64e00 00000100:00100000:2.0:1788572561.556767:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 356 00000100:00100000:2.0:1788572561.556779:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360628736:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572561.561954:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360628736:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 5162us (5437us total) trans 0 rc 32768/32768 00000100:00100000:2.0:1788572561.561970:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 356 00002000:00100000:2.0:1788572561.561987:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5a:0x0] (0->32767) req@ffff9627f5b64e00 x1875453360628736/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:222/0 lens 488/440 e 0 to 0 dl 1788572572 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572561.561998:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5b64e00 with x1875453360628736 ext(0->32767) 00002000:00010000:2.0:1788572561.562047:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627f5b64e00 00000100:00100000:1.0:1788572561.582236:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.582349:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360628864 00000100:00100000:0.0:1788572561.582368:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360628864 00002000:00100000:0.0:1788572561.582715:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5a:0x0] (32768->1048575) req@ffff9627f5b64a80 x1875453360628864/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:222/0 lens 488/0 e 0 to 0 dl 1788572572 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572561.582729:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5b64a80 with x1875453360628864 ext(32768->1048575) 00002000:00010000:0.0:1788572561.582736:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627f5b64a80 00000100:00100000:0.0:1788572561.582848:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 357 00000100:00100000:0.0:1788572561.582865:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2347:x1875453360628864:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:0.0:1788572561.619549:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2347:x1875453360628864:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 36674us (37306us total) trans 0 rc 1015808/1015808 00000100:00100000:0.0:1788572561.619570:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 357 00002000:00100000:0.0:1788572561.619588:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5a:0x0] (32768->1048575) req@ffff9627f5b64a80 x1875453360628864/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:222/0 lens 488/440 e 0 to 0 dl 1788572572 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572561.619599:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f5b64a80 with x1875453360628864 ext(32768->1048575) 00002000:00010000:0.0:1788572561.619607:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627f5b64a80 00000100:00100000:1.0:1788572561.631671:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.631869:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360629120 00000100:00100000:0.0:1788572561.631880:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360629120 00000100:00100000:0.0:1788572561.631964:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1050 00000100:00100000:0.0:1788572561.631977:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec03b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:36584:x1875453360629120:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:0.0:1788572561.632305:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec03b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:36584:x1875453360629120:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 331us (644us total) trans 4294970959 rc 0/0 00000100:00100000:0.0:1788572561.632315:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1050 00000100:00100000:2.0:1788572561.670754:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.670866:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360629248 00000100:00100000:0.0:1788572561.670877:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360629248 00000100:00100000:0.0:1788572561.670957:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4844 00000100:00100000:0.0:1788572561.670969:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:36586:x1875453360629248:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572561.670994:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572561.671045:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165835 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1521 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.671130:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.671147:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.671159:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572561.671196:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00000004:00200000:0.0:1788572561.671201:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572561.671218:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.671247:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165835 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 244 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.671266:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 243 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.671281:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:0.0:1788572561.671284:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572561.671288:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647e00, rc 0) 00000100:00100000:0.0:1788572561.671343:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36586:x1875453360629248:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 377us (599us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572561.671351:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4844 00000100:00100000:1.0:1788572561.685108:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.685230:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360629504 00000100:00100000:2.0:1788572561.685241:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360629504 00000100:00100000:2.0:1788572561.685290:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4845 00000100:00100000:2.0:1788572561.685300:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36586:x1875453360629504:12345-192.168.202.40@tcp:58:lfs.0 00000040:00080000:2.0:1788572561.685825:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].167, 136 off30696 00000100:00100000:2.0:1788572561.685920:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36586:x1875453360629504:12345-192.168.202.40@tcp:58:lfs.0 Request processed in 619us (819us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.685928:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4845 00000040:00080000:3.0:1788572561.724683:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572561.724694:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572561.724739:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 168 count 2 00000040:00080000:3.0:1788572561.724748:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572561.725388:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382397056:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572561.725525:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca87f000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572561.743734:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ead80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382397056:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572561.749564:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.749670:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360629632 00000100:00100000:2.0:1788572561.749681:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360629632 00000100:00100000:2.0:1788572561.749749:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4846 00000100:00100000:2.0:1788572561.749759:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36592:x1875453360629632:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572561.749791:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xa7:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572561.749928:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36592:x1875453360629632:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 169us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.749936:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4846 00000100:00100000:1.0:1788572561.754403:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.754502:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360629888 00000100:00100000:0.0:1788572561.754514:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360629888 00000100:00100000:0.0:1788572561.754597:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4847 00000100:00100000:0.0:1788572561.754610:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36592:x1875453360629888:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572561.754633:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572561.754655:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165843 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1528 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.754766:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16584a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.754782:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16584a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.754796:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16584a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572561.754818:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa7:0x0]: returning size 1048576 00010000:00010000:0.0:1788572561.754948:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165843 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1528 expref: 62 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.754973:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16584a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1528 expref: 61 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.754981:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:0.0:1788572561.754984:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572561.754987:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646200, rc 0) 00000100:00100000:0.0:1788572561.755218:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360629888:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 611us (824us total) trans 4294970960 rc 0/0 00000100:00100000:0.0:1788572561.755226:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4847 00000100:00100000:1.0:1788572561.761741:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572561.761845:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360630016 00000100:00100000:3.0:1788572561.761856:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360630016 00000100:00100000:3.0:1788572561.761929:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 373 00000100:00100000:3.0:1788572561.761941:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2348:x1875453360630016:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572561.761958:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572561.761985:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16584a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1528 expref: 61 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572561.762032:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:3.0:1788572561.762038:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572561.762110:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2348:x1875453360630016:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 172us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572561.762122:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 373 00000100:00100000:2.0:1788572561.766905:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.766974:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360630144 00000100:00100000:0.0:1788572561.766983:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360630144 00000100:00100000:0.0:1788572561.767095:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1407 00000100:00100000:0.0:1788572561.767106:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2347:x1875453360630144:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572561.767147:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767165:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572561.767169:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572561.767175:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767189:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767218:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767228:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767236:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767243:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572561.767247:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767255:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767269:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.767276:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb165858 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572561.767334:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2347:x1875453360630144:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 231us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572561.767340:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1407 00000100:00100000:1.0:1788572561.781788:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.781929:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360630400 00000100:00100000:2.0:1788572561.781945:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360630400 00000100:00100000:2.0:1788572561.782074:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4848 00000100:00100000:2.0:1788572561.782086:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36592:x1875453360630400:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572561.782229:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36592:x1875453360630400:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 145us (455us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.782237:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4848 00000100:00100000:1.0:1788572561.786081:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.786496:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360630528 00000100:00100000:0.0:1788572561.786511:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360630528 00000100:00100000:0.0:1788572561.786786:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1408 00000100:00100000:0.0:1788572561.786800:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2349:x1875453360630528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572561.786988:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572561.787040:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f152f expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.787079:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572561.787084:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572561.787094:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 10 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.787111:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 10 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.787125:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647400, rc 0) 00000100:00100000:0.0:1788572561.787180:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360630528:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 384us (1175us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572561.787186:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1408 00000100:00100000:3.0:1788572561.801146:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.801453:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360630784 00000100:00100000:2.0:1788572561.801464:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360630784 00000100:00100000:2.0:1788572561.801533:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4849 00000100:00100000:2.0:1788572561.801544:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36592:x1875453360630784:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572561.801675:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36592:x1875453360630784:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 133us (541us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.801684:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4849 00000100:00100000:1.0:1788572561.815826:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572561.816119:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360630912 00000100:00100000:3.0:1788572561.816131:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360630912 00002000:00100000:3.0:1788572561.816194:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5a:0x0] (0->1048575) req@ffff9627dfae6a00 x1875453360630912/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:222/0 lens 488/0 e 0 to 0 dl 1788572572 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572561.816206:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae6a00 with x1875453360630912 ext(0->1048575) 00002000:00010000:3.0:1788572561.816212:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfae6a00 00000100:00100000:3.0:1788572561.816245:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 358 00000100:00100000:3.0:1788572561.816256:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360630912:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1788572561.846220:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360630912:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 Request processed in 29954us (30392us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788572561.846236:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 358 00002000:00100000:3.0:1788572561.846249:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5a:0x0] (0->1048575) req@ffff9627dfae6a00 x1875453360630912/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:222/0 lens 488/440 e 0 to 0 dl 1788572572 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572561.846261:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfae6a00 with x1875453360630912 ext(0->1048575) 00002000:00010000:3.0:1788572561.846267:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627dfae6a00 00000100:00100000:1.0:1788572561.877932:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.878226:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360631040 00000100:00100000:2.0:1788572561.878238:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360631040 00000100:00100000:2.0:1788572561.878371:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4850 00000100:00100000:2.0:1788572561.878383:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36592:x1875453360631040:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572561.878415:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572561.878438:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb165866 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1536 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.878474:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.878489:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.878506:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.878614:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb165866 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 62 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.878633:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 61 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.878641:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:2.0:1788572561.878644:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572561.878649:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1800, rc 0) 00000100:00100000:2.0:1788572561.878730:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360631040:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 349us (807us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.878737:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4850 00000100:00100000:1.0:1788572561.884796:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.884928:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360631168 00000100:00100000:2.0:1788572561.884938:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360631168 00000100:00100000:2.0:1788572561.885023:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4851 00000100:00100000:2.0:1788572561.885035:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360631168:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572561.885079:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xa7:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572561.885194:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360631168:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 160us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.885201:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4851 00000100:00100000:1.0:1788572561.888607:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572561.888711:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360631296 00000100:00100000:0.0:1788572561.888731:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360631296 00000100:00100000:0.0:1788572561.888814:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1409 00000100:00100000:0.0:1788572561.888826:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360631296:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572561.888870:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888888:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572561.888892:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572561.888910:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888924:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888952:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888960:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888968:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888974:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572561.888979:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.888987:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.889001:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572561.889030:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000087e9f0ef/0x85ace67feb165874 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572561.889121:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360631296:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 298us (523us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572561.889130:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1409 00000100:00100000:1.0:1788572561.896111:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.896206:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360631552 00000100:00100000:2.0:1788572561.896217:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360631552 00000100:00100000:2.0:1788572561.896269:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4852 00000100:00100000:2.0:1788572561.896279:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360631552:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572561.896363:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.896382:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 61 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.896404:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.896439:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 61 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.896453:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 61 pid: 5837 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572561.896468:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b50380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382397184:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572561.896560:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572561.925128:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.926178:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360631680 00000100:00100000:2.0:1788572561.926192:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360631680 00010000:00010000:2.0:1788572561.926241:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 62 pid: 5837 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572561.926292:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1127 00000100:00100000:2.0:1788572561.926302:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360631680:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572561.926318:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572561.926325:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 62 pid: 5837 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.926339:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 62 pid: 5837 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.926361:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:2.0:1788572561.926366:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.926376:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.926384:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572561.926416:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572561.926469:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360631680:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 168us (1351us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572561.926476:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1127 00010000:00010000:1.0:1788572561.926985:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16586d lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1536 expref: 61 pid: 5837 timeout: 1009 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572561.927025:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b50380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382397184:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572561.927055:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927066:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927076:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927267:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927275:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927282:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927288:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572561.927292:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927300:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927316:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:1.0:1788572561.927318:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572561.927323:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572561.927328:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16587b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572561.927478:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360631552:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 31199us (31377us total) trans 4294970961 rc 0/0 00000100:00100000:1.0:1788572561.927488:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4852 00000040:00080000:2.0:1788572561.929594:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572561.929604:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572561.930219:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:2.0:1788572561.930228:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1788572561.932670:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572561.932767:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360631936 00000100:00100000:2.0:1788572561.932778:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360631936 00000100:00100000:2.0:1788572561.932840:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1051 00000100:00100000:2.0:1788572561.932851:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:36592:x1875453360631936:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572561.933124:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:36592:x1875453360631936:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 273us (460us total) trans 4294970962 rc 0/0 00000100:00100000:2.0:1788572561.933136:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1051 00000100:00100000:0.0:1788572562.031923:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572562.032706:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360632064 00000100:00100000:3.0:1788572562.032726:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360632064 00000100:00100000:3.0:1788572562.032803:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 389 00000100:00100000:3.0:1788572562.032815:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2348:x1875453360632064:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572562.032858:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572562.032877:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb165882 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f153d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.032948:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.032964:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.032978:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.032994:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033022:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033031:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033237:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033245:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033251:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033256:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572562.033260:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033272:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033284:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572562.033287:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572562.033292:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033298:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb165889 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033311:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033316:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033322:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033327:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572562.033329:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033334:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033341:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x3a81c6f5 (ffff9626c4b10700) 00010000:00010000:3.0:1788572562.033343:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572562.033345:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033354:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165890 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x3a81c6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033383:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb165882 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f153d expref: 63 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033392:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572562.033395:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572562.033399:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb165882 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f153d expref: 63 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.033409:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4e00, rc -2) 00000100:00100000:3.0:1788572562.033476:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2348:x1875453360632064:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 663us (1564us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572562.033484:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 389 00000100:00080000:0.0:1788572562.043251:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572562.545200:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572562.546102:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360632192 00000100:00100000:0.0:1788572562.546114:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360632192 00000100:00100000:0.0:1788572562.546184:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 390 00000100:00100000:0.0:1788572562.546196:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2348:x1875453360632192:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572562.546248:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572562.546269:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165897 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1544 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.546329:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 3/1,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.546344:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.546357:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.546373:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.546381:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.546389:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548792:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548808:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548816:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548822:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572562.548828:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548837:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548854:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:0.0:1788572562.548858:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572562.548863:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548870:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb16589e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548882:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548888:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548896:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548900:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572562.548903:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548909:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548915:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0xbaf9eef3 (ffff9627c5160c00) 00010000:00010000:0.0:1788572562.548918:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572562.548920:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548926:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1658a5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0xbaf9eef3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548971:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165897 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1544 expref: 63 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548981:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:0.0:1788572562.548984:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572562.548988:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165897 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1544 expref: 63 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.548997:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646e00, rc -2) 00000100:00100000:0.0:1788572562.549108:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2348:x1875453360632192:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2914us (3916us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572562.549121:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 390 00010000:00010000:1.0:1788572562.555489:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572562.753913:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788572562.753939:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382397312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788572562.754050:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572562.754121:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382397312 00000100:00100000:0.0:1788572562.754129:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382397312 00000100:00100000:0.0:1788572562.754190:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1788572562.754214:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1875453382397312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788572562.754251:0:6684:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1788572562.754293:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1875453382397312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572562.754300:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00080000:0.0:1788572562.754321:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572562.754345:0:3305:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967671 gen 2 00000004:00080000:0.0:1788572562.754351:0:3305:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ff0a8a80, transno 4294967670 00000004:00080000:0.0:1788572562.754367:0:3305:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627ff0a9500, transno 4294967671 00000100:00100000:0.0:1788572562.754379:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382397312:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1788572562.759000:0:23546:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572562.759058:0:23546:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572562.759106:0:23546:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 168 count 2 00000040:00080000:1.0:1788572562.759114:0:23546:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1788572562.856576:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572562.856742:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360632320 00000100:00100000:1.0:1788572562.856752:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360632320 00000100:00100000:1.0:1788572562.856810:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4853 00000100:00100000:1.0:1788572562.856822:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:36600:x1875453360632320:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:1.0:1788572562.856840:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572562.856861:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1658ac lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f154b expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.856935:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1658b3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.856955:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1658b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.856968:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1658b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572562.857035:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00000004:00200000:1.0:1788572562.857040:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572562.857061:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1658b3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.857090:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1658ac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f154b expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.857109:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1658b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f154b expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.857118:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:1.0:1788572562.857122:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572562.857126:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bce00, rc 0) 00000100:00100000:1.0:1788572562.857189:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:36600:x1875453360632320:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 370us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572562.857197:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4853 00000100:00100000:0.0:1788572562.888106:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572562.888225:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360632576 00000100:00100000:2.0:1788572562.888236:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360632576 00000100:00100000:2.0:1788572562.888289:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4854 00000100:00100000:2.0:1788572562.888298:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:36601:x1875453360632576:12345-192.168.202.40@tcp:36:rm.0 00010000:00010000:2.0:1788572562.888354:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572562.888382:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb16577f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14db expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572562.888402:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:2.0:1788572562.888413:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb1658b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f154b expref: 62 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572562.888422:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:2.0:1788572562.888426:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572562.888450:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572562.888464:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 242 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572562.888474:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 4/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572562.889646:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 242 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572562.889681:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 242 pid: 22536 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572562.889698:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b50000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382397440:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572562.889948:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2463000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572562.897297:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572562.898143:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360632704 00000100:00100000:0.0:1788572562.898154:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360632704 00010000:00010000:0.0:1788572562.898226:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 243 pid: 22536 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572562.898266:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1128 00000100:00100000:0.0:1788572562.898278:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:2349:x1875453360632704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572562.898295:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572562.898302:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 243 pid: 22536 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.898315:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 243 pid: 22536 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.898337:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:0.0:1788572562.898342:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.898352:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.898360:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.898382:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572562.898437:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:2349:x1875453360632704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 163us (1151us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572562.898445:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1128 00000100:00080000:0.0:1788572562.898465:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788572562.901631:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16583c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1521 expref: 242 pid: 22536 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572562.901658:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b50000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382397440:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572562.901683:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.901693:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.901700:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.901821:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.901836:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.901847:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.902879:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.902897:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 241 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.902908:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.902949:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 241 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.902965:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 241 pid: 23058 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572562.902980:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382397568:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572562.904114:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c533d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572562.907810:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572562.907949:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382397568:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572562.907977:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572562.910107:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360632832 00000100:00100000:0.0:1788572562.910123:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360632832 00010000:00010000:0.0:1788572562.910186:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 242 pid: 23058 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572562.910230:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1129 00000100:00100000:0.0:1788572562.910241:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:2349:x1875453360632832:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572562.910257:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572562.910265:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 242 pid: 23058 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.910279:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 242 pid: 23058 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.910305:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c3715600/0x85ace67feb1657ef lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14fe expref: 242 pid: 23058 timeout: 1010 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.910323:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:0.0:1788572562.910328:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.910338:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.910346:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.910380:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572562.910551:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2349:x1875453360632832:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 258us (2695us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572562.910561:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1129 00010000:00010000:3.0:1788572562.910620:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.910629:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572562.913662:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].76, 64 off14448 00000004:00080000:3.0:1788572562.913679:0:5838:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 910982674456ns, next at 911982670558ns, rc = 0 00010000:00010000:3.0:1788572562.914246:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914266:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914279:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914288:0:5838:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914295:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914302:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914318:0:5838:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788572562.914341:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914347:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914355:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914361:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572562.914365:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914373:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914385:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:3.0:1788572562.914389:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572562.914393:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914400:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb1658c8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914418:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914424:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914430:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914434:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572562.914436:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914445:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914453:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:3.0:1788572562.914455:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572562.914458:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.914463:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1658ba lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572562.914474:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000024478017 reply state 00000000c6129d49 transno 4294970963 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572562.914495:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788572562.914823:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572562.914830:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.914836:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572562.914839:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.914850:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.914859:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa7:0x0].0x0 (ffff9627c5160d00) 00010000:00010000:0.0:1788572562.914862:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572562.914865:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.914871:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1658cf lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.914885:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572562.914887:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572562.914889:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572562.915642:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:36601:x1875453360632576:12345-192.168.202.40@tcp:36:rm.0 Request processed in 27337us (27540us total) trans 4294970963 rc 0/0 00000100:00100000:3.0:1788572562.915654:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4854 00010000:00010000:0.0:1788572562.921933:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.921954:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.921962:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.921968:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572562.921972:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.921980:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.921997:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9626c4b10700) 00010000:00010000:0.0:1788572562.922619:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572562.922627:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572562.922636:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1658c1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572562.935403:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572562.935669:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360633088 00000100:00100000:1.0:1788572562.935682:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360633088 00000100:00100000:1.0:1788572562.935769:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4855 00000100:00100000:1.0:1788572562.935780:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:36602:x1875453360633088:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:1.0:1788572562.935808:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572562.935834:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1658d6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1552 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.935993:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.936053:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.936072:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.936101:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.936109:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.936116:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572562.937969:0:5839:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5b:0x0], index 0: 0 00010000:00010000:1.0:1788572562.938253:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938276:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938290:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938341:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938355:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938363:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938370:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572562.938376:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938385:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938400:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:1.0:1788572562.938404:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572562.938410:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938417:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1658dd lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572562.938433:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005e666688 reply state 000000002e818e12 transno 4294970964 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572562.938443:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1788572562.938484:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1658d6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 65 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938498:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572562.938888:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572562.938891:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572562.938894:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc 0) 00000100:00100000:1.0:1788572562.939092:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:36602:x1875453360633088:12345-192.168.202.40@tcp:101:dd.0 Request processed in 3305us (3694us total) trans 4294970964 rc 0/0 00000100:00100000:1.0:1788572562.939102:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4855 00010000:00010000:3.0:1788572562.948594:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.948612:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.948621:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.948628:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572562.948633:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.948648:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.948670:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627ff5ab000) 00010000:00010000:3.0:1788572562.948673:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572562.948679:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572562.948686:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb1658e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572562.977503:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572562.977540:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360633344 00000100:00100000:0.0:1788572562.977549:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360633344 00000100:00100000:0.0:1788572562.977618:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1052 00000100:00100000:0.0:1788572562.977628:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36602:x1875453360633344:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:0.0:1788572562.978072:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:36602:x1875453360633344:12345-192.168.202.40@tcp:35:dd.0 Request processed in 444us (577us total) trans 4294970965 rc 0/0 00000100:00100000:0.0:1788572562.978080:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1052 00000100:00100000:0.0:1788572563.058614:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572563.058800:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360633472 00000100:00100000:2.0:1788572563.058810:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360633472 00000100:00100000:2.0:1788572563.058875:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 391 00000100:00100000:2.0:1788572563.058885:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360633472:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572563.058924:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572563.058944:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1658f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1559 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.059542:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:2.0:1788572563.059621:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.059638:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.059652:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.059737:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.059747:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.059756:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060080:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060090:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060097:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060103:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572563.060108:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060117:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060131:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:3.0:1788572563.060134:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572563.060139:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060147:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb165900 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060160:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060166:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060173:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060178:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572563.060181:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060187:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060195:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x72ab5c99 (ffff9626c4b10e00) 00010000:00010000:3.0:1788572563.060197:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572563.060200:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060206:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb165907 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x72ab5c99 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060246:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1658f9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1559 expref: 66 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060256:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:3.0:1788572563.060259:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572563.060263:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1658f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1559 expref: 66 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.060273:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0e00, rc -2) 00000100:00100000:3.0:1788572563.060341:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360633472:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1456us (1735us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572563.060350:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 391 00000100:00100000:3.0:1788572563.060870:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360633600 00000100:00100000:3.0:1788572563.060878:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360633600 00000100:00100000:1.0:1788572563.062869:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 392 00000100:00100000:1.0:1788572563.062887:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2348:x1875453360633600:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572563.062927:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572563.062947:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16590e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1560 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063033:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063049:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063061:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063077:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063084:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063092:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063282:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063291:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063298:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063305:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.063310:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063318:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063330:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:1.0:1788572563.063333:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.063338:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063345:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165915 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063357:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063363:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063369:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063375:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.063379:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.063384:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.064609:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0xa712eb95 (ffff9627ff5abb00) 00010000:00010000:1.0:1788572563.064613:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.064618:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.064625:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16591c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0xa712eb95 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.064673:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16590e lrc: 1/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1560 expref: 66 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.064685:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:1.0:1788572563.064687:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572563.064691:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16590e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1560 expref: 66 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.064712:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bdc00, rc -2) 00000100:00100000:1.0:1788572563.064796:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2348:x1875453360633600:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1915us (5255us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572563.064806:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 392 00000100:00100000:0.0:1788572563.111101:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572563.111191:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360633856 00000100:00100000:1.0:1788572563.111201:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360633856 00000100:00100000:1.0:1788572563.111270:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4856 00000100:00100000:1.0:1788572563.111280:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:36605:x1875453360633856:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572563.111302:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572563.111325:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165923 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1567 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111418:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 3/1,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111437:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111452:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111468:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111474:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111480:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111763:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111774:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111781:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572563.111791:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa8:0x0]: returning size 0 00010000:00010000:1.0:1788572563.111899:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111907:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111914:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111920:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.111924:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111933:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111945:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:1.0:1788572563.111948:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.111953:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111960:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16592a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111976:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111982:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111991:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.111996:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.111998:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.112058:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.112067:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627ff5abb00) 00010000:00010000:1.0:1788572563.112070:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.112073:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.112078:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165931 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.112123:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165923 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 244 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.112138:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 243 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.112146:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572563.112148:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572563.112151:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bda00, rc 0) 00000100:00100000:1.0:1788572563.112287:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36605:x1875453360633856:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1007us (1197us total) trans 4294970966 rc 0/0 00000100:00100000:1.0:1788572563.112299:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4856 00000100:00100000:2.0:1788572563.133964:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.135901:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360634112 00000100:00100000:0.0:1788572563.135918:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360634112 00000100:00100000:0.0:1788572563.136058:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4857 00000100:00100000:0.0:1788572563.136070:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36605:x1875453360634112:12345-192.168.202.40@tcp:55:lfs.0 00010000:00010000:0.0:1788572563.136148:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.136165:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.136196:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 243 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.136205:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 5/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.136246:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 243 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.136261:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 243 pid: 22536 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.136276:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382397696:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572563.136393:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.136404:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 64 pid: 5839 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.136412:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382397824:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572563.136436:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d600 going to sleep for 11 seconds 00000100:00100000:2.0:1788572563.153866:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572563.153915:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572563.154040:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360634240 00000100:00100000:3.0:1788572563.154051:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360634240 00010000:00010000:3.0:1788572563.154092:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 244 pid: 22536 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572563.154121:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360634368 00000100:00100000:3.0:1788572563.154124:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360634368 00010000:00010000:3.0:1788572563.154138:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 65 pid: 5839 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572563.154160:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1130 00000100:00100000:3.0:1788572563.154170:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:2350:x1875453360634240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572563.154185:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572563.154191:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 244 pid: 22536 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154205:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 244 pid: 22536 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154229:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:3.0:1788572563.154235:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154244:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154250:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572563.154307:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:2350:x1875453360634240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 137us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.154314:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1130 00000100:00100000:3.0:1788572563.154358:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1131 00000100:00100000:3.0:1788572563.154365:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360634368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572563.154371:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572563.154377:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 65 pid: 5839 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154387:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 65 pid: 5839 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154404:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:3.0:1788572563.154408:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154417:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154425:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.154443:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572563.154460:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:2349:x1875453360634368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 96us (544us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.154466:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1131 00010000:00010000:0.0:1788572563.155412:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165938 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1567 expref: 243 pid: 22536 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.155431:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382397696:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572563.155447:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb1658eb lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1552 expref: 64 pid: 5839 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.155456:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382397824:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572563.155494:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157132:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157140:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157368:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157376:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157384:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157390:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572563.157395:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157404:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157421:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572563.157424:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572563.157430:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.157437:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165946 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.158669:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36605:x1875453360634112:12345-192.168.202.40@tcp:55:lfs.0 Request processed in 22592us (24707us total) trans 4294970967 rc 0/0 00000100:00100000:0.0:1788572563.158681:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4857 00000100:00100000:2.0:1788572563.168263:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.171179:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360634624 00000100:00100000:0.0:1788572563.171197:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360634624 00000100:00100000:2.0:1788572563.174961:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1053 00000100:00100000:2.0:1788572563.174980:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:36605:x1875453360634624:12345-192.168.202.40@tcp:35:lfs.0 00000100:00100000:2.0:1788572563.175293:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:36605:x1875453360634624:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 316us (7038us total) trans 4294970968 rc 0/0 00000100:00100000:2.0:1788572563.175303:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1053 00000100:00100000:3.0:1788572563.201173:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572563.203110:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360634752 00000100:00100000:1.0:1788572563.203127:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360634752 00000100:00100000:1.0:1788572563.203252:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4858 00000100:00100000:1.0:1788572563.203264:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:36606:x1875453360634752:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572563.203299:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572563.203325:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16594d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f156e expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203445:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203461:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203482:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203498:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203505:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203511:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203609:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203622:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203629:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572563.203676:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa8:0x0]: returning size 0 00000004:00200000:1.0:1788572563.203681:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572563.203711:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203722:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203730:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203737:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203744:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203750:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.203755:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203763:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203774:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:1.0:1788572563.203778:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.203783:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203790:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165954 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203802:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203809:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203815:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203820:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.203823:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203829:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203836:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627ff5abb00) 00010000:00010000:1.0:1788572563.203838:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.203841:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203846:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb16595b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203877:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb16594d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 246 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203890:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 245 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.203898:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:1.0:1788572563.203900:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572563.203902:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc000, rc 0) 00000100:00100000:1.0:1788572563.203971:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:36606:x1875453360634752:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 710us (2808us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572563.203979:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4858 00000100:00100000:3.0:1788572563.209896:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.209973:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360634880 00000100:00100000:0.0:1788572563.209984:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360634880 00000100:00080000:2.0:1788572563.210134:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572563.210145:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1410 00000100:00100000:0.0:1788572563.210169:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2349:x1875453360634880:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572563.210188:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572563.210211:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1575 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572563.210240:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572563.210259:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=911s v=(0 0) 00010000:00010000:0.0:1788572563.210264:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572563.210273:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 72 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.210292:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 72 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.210311:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646800, rc 0) 00000100:00100000:0.0:1788572563.210373:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2349:x1875453360634880:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 219us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.210380:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1410 00000100:00100000:2.0:1788572563.240675:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.240825:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360635008 00000100:00100000:0.0:1788572563.240835:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360635008 00000100:00100000:0.0:1788572563.240921:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4859 00000100:00100000:0.0:1788572563.240936:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:36606:x1875453360635008:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572563.240961:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572563.240985:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb165970 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f157c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.241228:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.241249:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.241265:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572563.241304:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xce:0x0]: returning size 16384 00000004:00200000:0.0:1788572563.241308:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572563.241329:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.241358:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb165970 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 247 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.241378:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 246 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.241401:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:0.0:1788572563.241405:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572563.241409:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647a00, rc 0) 00000100:00100000:0.0:1788572563.241478:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:36606:x1875453360635008:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 546us (813us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.241486:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4859 00000100:00100000:2.0:1788572563.254286:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572563.255141:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360635264 00000100:00100000:3.0:1788572563.255154:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360635264 00000100:00100000:3.0:1788572563.255215:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4860 00000100:00100000:3.0:1788572563.255226:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:36606:x1875453360635264:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:3.0:1788572563.255251:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572563.255277:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb16597e lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1583 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.255457:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165985 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.255473:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165985 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.255487:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165985 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572563.255512:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa8:0x0]: returning size 0 00010000:00010000:3.0:1788572563.255640:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb16597e lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1583 expref: 249 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.255662:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165985 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1583 expref: 248 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.255672:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:3.0:1788572563.255675:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1788572563.255679:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4200, rc 0) 00000100:00100000:3.0:1788572563.255888:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:36606:x1875453360635264:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 662us (1611us total) trans 4294970969 rc 0/0 00000100:00100000:3.0:1788572563.255897:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4860 00000100:00100000:3.0:1788572563.258230:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788572563.261109:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382398336:0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261189:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572563.261247:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3304:1875453382397952:0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261275:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572563.261314:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382398336 00000100:00100000:3.0:1788572563.261323:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382398336 00000100:00100000:3.0:1788572563.261368:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:3.0:1788572563.261376:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382398336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261427:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382398336:12345-0@lo:400:kworker.0 Request processed in 50us (238us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.261435:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:3.0:1788572563.261459:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382397952 00000100:00100000:3.0:1788572563.261462:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382397952 00000100:00100000:3.0:1788572563.261682:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:3.0:1788572563.261690:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3304:x1875453382397952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261721:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3304:x1875453382397952:12345-0@lo:400:kworker.0 Request processed in 33us (445us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.261727:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:3.0:1788572563.261781:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382398336:0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261791:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3304:1875453382397952:0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261808:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788572563.261814:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382398464:0@lo:400:kworker.0 00000100:00100000:0.0:1788572563.261831:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382398080:0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.261839:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572563.261887:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3303:1875453382398208:0@lo:400:kworker.0 00000100:00100000:2.0:1788572563.262093:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382398464 00000100:00100000:2.0:1788572563.262099:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382398464 00000100:00100000:2.0:1788572563.262134:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:2.0:1788572563.262143:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382398464:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1788572563.265354:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572563.265552:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572563.265627:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382398208 00000100:00100000:0.0:1788572563.265637:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382398208 00000100:00100000:1.0:1788572563.265653:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572563.265676:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572563.265703:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382398592:0@lo:400:kworker.0 00000100:00100000:1.0:1788572563.265734:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1788572563.265773:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572563.265891:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382398464:12345-0@lo:400:kworker.0 Request processed in 3749us (4048us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572563.265901:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:3.0:1788572563.266062:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382398464:0@lo:400:kworker.0 00000100:00100000:2.0:1788572563.266145:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382398080 00000100:00100000:2.0:1788572563.266150:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382398080 00000100:00100000:0.0:1788572563.266169:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:0.0:1788572563.266181:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1875453382398208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.266211:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360635392 00000100:00100000:3.0:1788572563.266215:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360635392 00000100:00100000:0.0:1788572563.266232:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1875453382398208:12345-0@lo:400:kworker.0 Request processed in 55us (684us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.266238:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:3.0:1788572563.266238:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382398592 00000100:00100000:3.0:1788572563.266240:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382398592 00000100:00100000:0.0:1788572563.266249:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1788572563.266257:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382398080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.266262:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1788572563.266268:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1875453382398592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788572563.266289:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382398080:12345-0@lo:400:kworker.0 Request processed in 34us (612us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.266295:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1875453382398592:12345-0@lo:400:kworker.0 Request processed in 28us (560us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.266296:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1788572563.266301:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:1.0:1788572563.266356:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382398080:0@lo:400:kworker.0 00000100:00100000:1.0:1788572563.266367:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382398592:0@lo:400:kworker.0 00000100:00100000:3.0:1788572563.266411:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1411 00000100:00100000:3.0:1788572563.266422:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2349:x1875453360635392:12345-192.168.202.40@tcp:1:lfs.0 00010000:00010000:3.0:1788572563.266457:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.266487:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 72 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.266503:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572563.266508:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572563.266513:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.266546:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 72 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.266560:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 72 pid: 13272 timeout: 1011 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572563.266573:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382398720:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572563.266627:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00080000:3.0:1788572563.266638:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788572563.270253:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3303:1875453382398208:0@lo:400:kworker.0 00000100:00100000:2.0:1788572563.272380:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572563.272501:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382398720:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572563.272527:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.273097:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360635520 00000100:00100000:0.0:1788572563.273107:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360635520 00010000:00010000:0.0:1788572563.273159:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 73 pid: 13272 timeout: 1011 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.273203:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1132 00000100:00100000:0.0:1788572563.273215:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360635520:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572563.273229:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572563.273253:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 73 pid: 13272 timeout: 1011 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.273267:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 73 pid: 13272 timeout: 1011 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.273291:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d20c2b5d/0x85ace67feb165969 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1575 expref: 73 pid: 13272 timeout: 1011 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.273309:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572563.273314:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572563.273321:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.273332:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.273376:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572563.273438:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2349:x1875453360635520:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 227us (1070us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.273447:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1132 00010000:00010000:2.0:1788572563.274042:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274054:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274081:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274095:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274102:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274109:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572563.274113:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274123:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274137:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.274145:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b550ad7/0x85ace67feb165993 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572563.274203:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2349:x1875453360635392:12345-192.168.202.40@tcp:1:lfs.0 Request processed in 7783us (8551us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572563.274211:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1411 00000100:00100000:2.0:1788572563.279306:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.279381:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360635776 00000100:00100000:0.0:1788572563.279391:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360635776 00000100:00100000:0.0:1788572563.279475:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1054 00000100:00100000:0.0:1788572563.279487:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:36606:x1875453360635776:12345-192.168.202.40@tcp:35:lfs.0 00000004:00010000:0.0:1788572563.279566:0:5840:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0xa8:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165985 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1583 expref: 248 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572563.279794:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970970 00000001:00080000:0.0:1788572563.280506:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970971 00010000:00010000:0.0:1788572563.280574:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.280591:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 251 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.280602:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.280640:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 251 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.280658:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 251 pid: 23058 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.280681:0:5840:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5840:1875453382398848:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572563.280786:0:5840:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572563.287449:0:5840:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5840:1875453382398848:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572563.287495:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572563.288720:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572563.289061:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360635904 00000100:00100000:3.0:1788572563.289070:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360635904 00010000:00010000:3.0:1788572563.289107:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 252 pid: 23058 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572563.289147:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1133 00000100:00100000:3.0:1788572563.289157:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:2350:x1875453360635904:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572563.289172:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572563.289179:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 252 pid: 23058 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.289192:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 252 pid: 23058 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.289215:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165962 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f156e expref: 252 pid: 23058 timeout: 1011 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.289232:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:3.0:1788572563.289237:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.289247:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.289255:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572563.289275:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572563.289332:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:2350:x1875453360635904:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 176us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.289343:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1133 00010000:00010000:0.0:1788572563.289700:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.289710:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572563.289862:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970972 00000001:00080000:0.0:1788572563.290115:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970973 00000001:00080000:0.0:1788572563.290189:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970974 00000100:00100000:0.0:1788572563.290284:0:5840:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5840:1875453382398976:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1788572563.290341:0:5840:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572563.290377:0:5840:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c9932780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572563.290887:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382398976 00000100:00100000:3.0:1788572563.290895:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382398976 00000100:00100000:3.0:1788572563.290940:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:3.0:1788572563.290949:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5840:x1875453382398976:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1788572563.291070:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5840:x1875453382398976:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 122us (728us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572563.291078:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1788572563.291337:0:5840:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f95880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5840:1875453382398976:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1788572563.291436:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970975 00010000:00010000:2.0:1788572563.291489:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.291499:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.291507:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.291513:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572563.291518:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.291526:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.291542:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:2.0:1788572563.291545:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1788572563.291550:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.291557:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16599a lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572563.291759:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0xcf:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1788572563.291843:0:5840:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].79, 64 off14640 00000004:00080000:2.0:1788572563.291915:0:5840:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 911360909959ns, next at 912360907488ns, rc = 0 00000001:00080000:2.0:1788572563.291923:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970976 00010000:00010000:2.0:1788572563.291960:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:2.0:1788572563.291963:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1788572563.291969:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb165985 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1583 expref: 255 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572563.292175:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:36606:x1875453360635776:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 12690us (12879us total) trans 4294970977 rc 0/0 00000100:00100000:2.0:1788572563.292184:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1054 00000020:00080000:2.0:1788572563.292188:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1788572563.567237:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572563.567334:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360636032 00000100:00100000:1.0:1788572563.567344:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360636032 00000100:00100000:1.0:1788572563.567420:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 394 00000100:00100000:1.0:1788572563.567430:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2348:x1875453360636032:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572563.567476:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572563.567663:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1659a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f158a expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.567733:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.567760:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.567772:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.567788:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.567795:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.567802:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568064:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568072:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568078:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568083:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.568088:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568099:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568112:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:1.0:1788572563.568115:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.568120:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568127:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568140:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568146:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568153:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568158:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.568161:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568167:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568173:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x343c220d (ffff9627ff5abb00) 00010000:00010000:1.0:1788572563.568176:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.568179:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568189:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x343c220d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568221:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1659a1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f158a expref: 65 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568231:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:1.0:1788572563.568233:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572563.568237:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1659a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f158a expref: 65 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.568246:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bdc00, rc -2) 00000100:00100000:1.0:1788572563.568324:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2348:x1875453360636032:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 894us (1097us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572563.568332:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 394 00010000:00010000:2.0:1788572563.581145:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572563.698336:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572563.698435:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360636288 00000100:00100000:1.0:1788572563.698445:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360636288 00000100:00100000:1.0:1788572563.698522:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4861 00000100:00100000:1.0:1788572563.698533:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:36628:x1875453360636288:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:1.0:1788572563.698558:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572563.698580:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1659b6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1591 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.698657:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 3/1,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.698674:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.698687:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.698703:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.698710:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.698717:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699101:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1659cb lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699111:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1659cb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699118:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1659cb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572563.699128:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa8:0x0]: returning size 0 00010000:00010000:1.0:1788572563.699227:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699236:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699242:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699248:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.699253:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699261:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699273:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:1.0:1788572563.699276:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.699281:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699288:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb1659bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699302:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699308:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699314:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699318:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572563.699321:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699327:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699335:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xe6e8db52 (ffff9627ff5abb00) 00010000:00010000:1.0:1788572563.699337:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572563.699341:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699345:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb1659c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0xe6e8db52 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699389:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb1659b6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1591 expref: 257 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699403:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1659cb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1591 expref: 256 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572563.699410:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572563.699412:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572563.699415:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bca00, rc 0) 00000100:00100000:1.0:1788572563.701208:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:36628:x1875453360636288:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 2667us (2871us total) trans 4294970978 rc 0/0 00000100:00100000:1.0:1788572563.701220:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4861 00000100:00100000:1.0:1788572563.705972:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572563.706100:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360636544 00000100:00100000:2.0:1788572563.706111:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360636544 00000100:00100000:2.0:1788572563.706176:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 374 00000100:00100000:2.0:1788572563.706187:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:36628:x1875453360636544:12345-192.168.202.40@tcp:103:md5sum.0 00010000:00010000:2.0:1788572563.706203:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572563.706228:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb1659cb lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1591 expref: 256 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572563.706246:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:2.0:1788572563.706257:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572563.706316:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:36628:x1875453360636544:12345-192.168.202.40@tcp:103:md5sum.0 Request processed in 131us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572563.706323:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 374 00000100:00100000:2.0:1788572563.710254:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.710354:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360636672 00000100:00100000:0.0:1788572563.710365:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360636672 00000100:00100000:0.0:1788572563.710462:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4862 00000100:00100000:0.0:1788572563.710483:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:36628:x1875453360636672:12345-192.168.202.40@tcp:58:md5sum.0 00010000:00010000:0.0:1788572563.710603:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.710622:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.710635:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572563.710808:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].168, 136 off30832 00000100:00100000:0.0:1788572563.710913:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:36628:x1875453360636672:12345-192.168.202.40@tcp:58:md5sum.0 Request processed in 432us (661us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.710921:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4862 00000100:00100000:3.0:1788572563.713646:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.713760:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360636800 00000100:00100000:0.0:1788572563.713771:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360636800 00000100:00100000:0.0:1788572563.713864:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4863 00000100:00100000:0.0:1788572563.713874:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:36628:x1875453360636800:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572563.713899:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572563.713925:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1598 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.713995:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.714050:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 4/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 256 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.714070:0:9040:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.714087:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 256 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.714104:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627c5160400) 00010000:00010000:0.0:1788572563.714109:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572563.714114:0:9040:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 256 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.714123:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647000, rc 0) 00000100:00100000:0.0:1788572563.714182:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:36628:x1875453360636800:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 310us (547us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.714190:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4863 00000100:00100000:0.0:1788572563.771180:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5968700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382399104:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788572563.771253:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572563.771416:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382399104 00000100:00100000:1.0:1788572563.771425:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382399104 00000100:00100000:1.0:1788572563.771833:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1788572563.771844:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382399104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788572563.771881:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 193789952 pending 0 free 3855613952 avail 3853516800 00000100:00100000:1.0:1788572563.771942:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382399104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (689us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572563.771949:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:2.0:1788572563.772050:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5968700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382399104:0@lo:13:osp-pre-1-0.0 00000040:00080000:2.0:1788572563.963033:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572563.963054:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572563.963103:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 169 count 3 00000040:00080000:2.0:1788572563.963111:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1788572563.963259:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcd500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382399232:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572563.963407:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8900 going to sleep for 11 seconds 00000100:00100000:2.0:1788572563.971296:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcd500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382399232:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572563.972188:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.972415:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360636928 00000100:00100000:0.0:1788572563.972423:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360636928 00000100:00100000:0.0:1788572563.972502:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4864 00000100:00100000:0.0:1788572563.972511:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360636928:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572563.972540:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xa8:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572563.972640:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360636928:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 130us (456us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.972648:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4864 00000100:00100000:3.0:1788572563.979225:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.979455:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360637056 00000100:00100000:0.0:1788572563.979472:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360637056 00000100:00100000:0.0:1788572563.979549:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4865 00000100:00100000:0.0:1788572563.979562:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360637056:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572563.979642:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979659:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979678:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979693:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979700:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979706:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979730:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979738:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979743:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572563.979785:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa8:0x0]: returning size 0 00000004:00200000:0.0:1788572563.979789:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572563.979801:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979811:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979818:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979830:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979836:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572563.979840:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979847:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979862:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572563.979865:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572563.979871:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979878:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1659e7 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.979892:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980397:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980404:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980410:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572563.980413:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980425:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980434:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x147832 (ffff9627c5160d00) 00010000:00010000:0.0:1788572563.980446:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572563.980449:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980455:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1659ee lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x147832 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980567:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980575:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980580:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980586:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572563.980589:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980595:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980603:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627c5160400) 00010000:00010000:0.0:1788572563.980605:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572563.980609:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.980618:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659f5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572563.980813:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360637056:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1252us (1597us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572563.980822:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4865 00000100:00100000:2.0:1788572563.992826:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572563.992910:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360637312 00000100:00100000:0.0:1788572563.992921:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360637312 00000100:00100000:0.0:1788572563.993046:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4866 00000100:00100000:0.0:1788572563.993060:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360637312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572563.993084:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572563.993105:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659fc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f159f expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993214:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993229:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993242:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572563.993254:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa8:0x0]: returning size 0 00010000:00010000:0.0:1788572563.993366:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993374:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993385:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993391:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572563.993395:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993403:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993415:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627c5160400) 00010000:00010000:0.0:1788572563.993418:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572563.993422:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993428:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a03 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa8:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993906:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659fc lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f159f expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993919:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572563.993922:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572563.993926:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1659fc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f159f expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572563.993936:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646600, rc 0) 00000100:00100000:0.0:1788572563.994071:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:36630:x1875453360637312:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1015us (1256us total) trans 4294970979 rc 301/301 00000100:00100000:0.0:1788572563.994082:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4866 00000100:00100000:0.0:1788572564.003478:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.003821:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360637440 00000100:00100000:1.0:1788572564.003831:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360637440 00000100:00100000:1.0:1788572564.003910:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4867 00000100:00100000:1.0:1788572564.003921:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:36630:x1875453360637440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572564.003956:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xa8:0x0] from 18446744073709551615 #0 00000100:00080000:3.0:1788572564.004079:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572564.004102:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:36630:x1875453360637440:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 183us (645us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.004110:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4867 00000100:00100000:3.0:1788572564.011194:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.021871:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360637696 00000100:00100000:1.0:1788572564.021886:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360637696 00000100:00100000:1.0:1788572564.021963:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4868 00000100:00100000:1.0:1788572564.021983:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:36630:x1875453360637696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572564.022027:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572564.022052:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a11 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15a6 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.022580:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165a18 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.022596:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165a18 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.022609:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165a18 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572564.022621:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa9:0x0]: returning size 0 00010000:00010000:1.0:1788572564.022668:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a11 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15a6 expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.022686:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165a18 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15a6 expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.022696:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:1.0:1788572564.022700:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572564.022704:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bde00, rc 0) 00000100:00100000:1.0:1788572564.024091:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:36630:x1875453360637696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2100us (12896us total) trans 4294970980 rc 0/0 00000100:00100000:1.0:1788572564.024103:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4868 00000100:00100000:2.0:1788572564.030164:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.030261:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360637952 00000100:00100000:3.0:1788572564.030272:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360637952 00000100:00100000:3.0:1788572564.030344:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4869 00000100:00100000:3.0:1788572564.030354:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:36630:x1875453360637952:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572564.030419:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572564.030447:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165a18 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15a6 expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.030538:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:3.0:1788572564.030551:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572564.030577:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.030591:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 255 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.030604:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.030635:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 255 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.030648:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 255 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.030661:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d88e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382399360:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572564.031132:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.042704:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.042865:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d88e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382399360:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572564.042900:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.043093:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360638080 00000100:00100000:2.0:1788572564.043101:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360638080 00010000:00010000:2.0:1788572564.043153:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 256 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.043184:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1134 00000100:00100000:2.0:1788572564.043193:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+256:2349:x1875453360638080:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572564.043208:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572564.043215:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 256 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.043228:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 256 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.043248:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647a00/0x85ace67feb165977 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xce:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f157c expref: 256 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.043262:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:2.0:1788572564.043266:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.043281:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.043289:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.043307:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572564.043353:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+255:2349:x1875453360638080:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 162us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.043360:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1134 00010000:00010000:3.0:1788572564.043501:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043509:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043533:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043543:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043550:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043622:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043629:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043636:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043641:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572564.043646:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043653:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043662:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:3.0:1788572564.043664:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572564.043668:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043674:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165a26 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043684:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043690:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043696:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043702:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572564.043705:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043711:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043718:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0xc37234b7 (ffff9626c4b10000) 00010000:00010000:3.0:1788572564.043720:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572564.043723:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.043729:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165a2d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xce:0x0].0xc37234b7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.043793:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:36630:x1875453360637952:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 13440us (13638us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572564.043801:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4869 00000100:00080000:1.0:1788572564.044184:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572564.050403:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572564.050495:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360638208 00000100:00100000:2.0:1788572564.050506:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360638208 00000100:00100000:2.0:1788572564.050571:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4870 00000100:00100000:2.0:1788572564.050581:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:36630:x1875453360638208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572564.050603:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572564.050628:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb165a34 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15ad expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.050666:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.050681:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.050693:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.050805:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb165a34 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 68 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.050833:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.050843:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:2.0:1788572564.050847:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572564.050851:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1400, rc 0) 00000100:00100000:2.0:1788572564.050936:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:36630:x1875453360638208:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 356us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.050943:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4870 00000100:00100000:2.0:1788572564.054946:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.055055:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360638464 00000100:00100000:0.0:1788572564.055067:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360638464 00000100:00100000:0.0:1788572564.055149:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4871 00000100:00100000:0.0:1788572564.055163:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:36630:x1875453360638464:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572564.055276:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.055296:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.055309:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572564.055546:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a39e7eb2 reply state 00000000d05cd077 transno 4294970981 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.055555:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572564.055659:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:36630:x1875453360638464:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 499us (725us total) trans 4294970981 rc 0/0 00000100:00100000:0.0:1788572564.055668:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4871 00010000:00010000:0.0:1788572564.057725:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.057743:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.057750:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.057757:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572564.057762:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.057770:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.057787:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:0.0:1788572564.057790:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788572564.057795:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.057802:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a42 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.059691:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.059765:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360638592 00000100:00100000:0.0:1788572564.059777:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360638592 00000100:00100000:0.0:1788572564.059849:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4872 00000100:00100000:0.0:1788572564.059859:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:36630:x1875453360638592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572564.059880:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572564.059908:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15b4 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.059973:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 69 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.059987:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 69 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.060031:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:0.0:1788572564.060034:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572564.060038:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646800, rc 0) 00000100:00100000:0.0:1788572564.060083:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:36630:x1875453360638592:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 226us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.060089:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4872 00000100:00100000:3.0:1788572564.066371:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572564.066475:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360638720 00000100:00100000:2.0:1788572564.066486:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360638720 00000100:00100000:2.0:1788572564.066549:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1055 00000100:00100000:2.0:1788572564.066560:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec039050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:36630:x1875453360638720:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 00000100:00100000:2.0:1788572564.066732:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec039050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:36630:x1875453360638720:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 Request processed in 173us (375us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.066740:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1055 00000100:00100000:1.0:1788572564.070378:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572564.071602:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360638976 00000100:00100000:2.0:1788572564.071616:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360638976 00000100:00100000:2.0:1788572564.071697:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1056 00000100:00100000:2.0:1788572564.071708:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:36630:x1875453360638976:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572564.072194:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:36630:x1875453360638976:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 486us (1826us total) trans 4294970982 rc 0/0 00000100:00100000:2.0:1788572564.072206:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1056 00000100:00100000:1.0:1788572564.081957:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572564.082761:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360639232 00000100:00100000:2.0:1788572564.082773:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360639232 00000100:00100000:2.0:1788572564.082844:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4873 00000100:00100000:2.0:1788572564.082854:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:36630:x1875453360639232:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572564.082998:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:36630:x1875453360639232:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 145us (1054us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.083558:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4873 00000100:00100000:3.0:1788572564.087905:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.088210:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360639488 00000100:00100000:1.0:1788572564.088220:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360639488 00000100:00100000:1.0:1788572564.088286:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4874 00000100:00100000:1.0:1788572564.088297:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:36630:x1875453360639488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572564.088320:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572564.088345:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15bb expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.088641:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.088657:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.088732:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.088773:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.088787:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 71 pid: 9040 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572564.088803:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382399488:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572564.088906:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982de40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.105995:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382399488:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572564.106170:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.110577:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.112990:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360639616 00000100:00100000:1.0:1788572564.113030:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360639616 00010000:00010000:2.0:1788572564.116080:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 72 pid: 9040 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.119179:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1135 00000100:00100000:2.0:1788572564.119207:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2349:x1875453360639616:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572564.119231:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572564.119243:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 72 pid: 9040 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.119260:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 72 pid: 9040 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.119283:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb165a49 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15b4 expref: 72 pid: 9040 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.119301:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:2.0:1788572564.119307:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.119316:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.119323:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.119361:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572564.119433:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2349:x1875453360639616:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 241us (8865us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.119442:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1135 00010000:00010000:0.0:1788572564.119522:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.119530:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572564.119549:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa9:0x0]: returning size 0 00000004:00080000:0.0:1788572564.119768:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x5b:0x0], index 1: 0 00010000:00010000:0.0:1788572564.119933:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.119941:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.119947:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.119953:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572564.119957:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.119975:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.119990:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:0.0:1788572564.119993:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572564.119997:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.120036:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165a57 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.120093:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a50 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15bb expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.120104:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xce:0x0].0x0 (ffff9627ffc19d00) 00010000:00010000:0.0:1788572564.120107:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572564.120111:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xce:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15bb expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.120120:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bdc00, rc 0) 00000100:00100000:0.0:1788572564.120155:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:36630:x1875453360639488:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 31861us (32273us total) trans 4294970983 rc 301/301 00000100:00100000:0.0:1788572564.120163:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4874 00000100:00100000:3.0:1788572564.128071:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.128253:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360639872 00000100:00100000:0.0:1788572564.128264:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360639872 00000100:00100000:0.0:1788572564.128340:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1057 00000100:00100000:0.0:1788572564.128353:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:36630:x1875453360639872:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572564.129109:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:36630:x1875453360639872:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 758us (1048us total) trans 4294970984 rc 0/0 00000100:00100000:0.0:1788572564.129122:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1057 00000100:00100000:1.0:1788572564.133046:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.133550:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640000 00000100:00100000:0.0:1788572564.133561:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640000 00000100:00100000:0.0:1788572564.134048:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4875 00000100:00100000:0.0:1788572564.134062:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:36630:x1875453360640000:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572564.134090:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572564.134112:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a65 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15c2 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.134229:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb165a6c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.134245:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb165a6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.134257:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb165a6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572564.134293:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa9:0x0]: returning size 0 00000004:00200000:0.0:1788572564.134298:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572564.134317:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb165a6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.134343:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165a65 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c2 expref: 73 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.134361:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb165a6c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c2 expref: 72 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.134375:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:0.0:1788572564.134378:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572564.134382:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647e00, rc 0) 00000100:00100000:0.0:1788572564.134454:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:36630:x1875453360640000:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 397us (1456us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.134462:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4875 00000100:00100000:3.0:1788572564.146037:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.146120:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640128 00000100:00100000:1.0:1788572564.146130:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640128 00000100:00100000:1.0:1788572564.146204:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1412 00000100:00100000:1.0:1788572564.146216:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360640128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572564.146236:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572564.146258:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15c9 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572564.146285:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1788572564.146305:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=912s v=(0 0) 00010000:00010000:1.0:1788572564.146309:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572564.146321:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 11 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.146341:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 11 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.146355:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc600, rc 0) 00000100:00100000:1.0:1788572564.146413:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360640128:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 200us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.146419:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1412 00000100:00080000:3.0:1788572564.146940:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572564.152759:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.152837:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640384 00000100:00100000:0.0:1788572564.152846:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640384 00000100:00100000:0.0:1788572564.152922:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4876 00000100:00100000:0.0:1788572564.152936:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:36630:x1875453360640384:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572564.153120:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:36630:x1875453360640384:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 187us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.153128:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4876 00000100:00100000:1.0:1788572564.165862:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.166069:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640512 00000100:00100000:3.0:1788572564.166080:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640512 00000100:00100000:3.0:1788572564.166146:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1413 00000100:00100000:3.0:1788572564.166156:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:36630:x1875453360640512:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572564.166176:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572564.166194:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15d0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.166228:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.166237:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572564.166242:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572564.166247:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: --/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.166285:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 12 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.166299:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 12 pid: 6681 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.166313:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b50000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382399616:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572564.166425:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c97cb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.176391:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572564.176674:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640640 00000100:00100000:2.0:1788572564.176684:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640640 00010000:00010000:2.0:1788572564.176733:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 13 pid: 6681 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.176778:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1136 00000100:00100000:2.0:1788572564.176788:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360640640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572564.176803:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572564.176826:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 13 pid: 6681 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.176840:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 13 pid: 6681 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.176863:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572564.176868:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572564.176877:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.176887:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.176925:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.176939:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382399744:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572564.177051:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b83c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788572564.177620:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b823db7e/0x85ace67feb165a73 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c9 expref: 13 pid: 6681 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.177637:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b50000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13272:1875453382399616:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572564.177669:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.177682:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5200, rc 0) 00000100:00100000:0.0:1788572564.177777:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:36630:x1875453360640512:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 11622us (11928us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.177790:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1413 00000100:00100000:2.0:1788572564.189742:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5823:1875453382399744:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572564.189773:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572564.189837:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efc850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360640640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 13049us (13455us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.189849:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1136 00000100:00100000:1.0:1788572564.216177:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.218556:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640768 00000100:00100000:0.0:1788572564.218573:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640768 00002000:00100000:0.0:1788572564.219078:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5b:0x0] (0->1048575) req@ffff9627fb471500 x1875453360640768/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:225/0 lens 488/0 e 0 to 0 dl 1788572575 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572564.219090:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb471500 with x1875453360640768 ext(0->1048575) 00002000:00010000:0.0:1788572564.219097:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb471500 00000100:00100000:2.0:1788572564.219281:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 359 00000100:00100000:2.0:1788572564.219298:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360640768:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572564.261232:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360640768:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 41925us (45047us total) trans 4294967670 rc 0/0 00000100:00100000:3.0:1788572564.261248:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 359 00002000:00100000:3.0:1788572564.261265:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5b:0x0] (0->1048575) req@ffff9627fb471500 x1875453360640768/t4294967670(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:225/0 lens 488/448 e 0 to 0 dl 1788572575 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572564.261281:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb471500 with x1875453360640768 ext(0->1048575) 00002000:00010000:3.0:1788572564.261288:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627fb471500 00000100:00100000:2.0:1788572564.271616:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.274027:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360640896 00000100:00100000:0.0:1788572564.274045:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360640896 00000100:00100000:0.0:1788572564.274203:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4877 00000100:00100000:0.0:1788572564.274215:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:36630:x1875453360640896:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:1.0:1788572564.324327:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:36630:x1875453360640896:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 50101us (52705us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.324343:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4877 00000001:00080000:1.0:1788572564.324379:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970984 is committed 00000001:00080000:1.0:1788572564.324396:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970975 is committed 00000001:00080000:1.0:1788572564.324424:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970978 is committed 00000004:00080000:1.0:1788572564.324794:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 47 committed 00000100:00100000:2.0:1788572564.325098:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572564.325116:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382399872:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1788572564.325172:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572564.325277:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382399872 00000100:00100000:0.0:1788572564.325285:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382399872 00000100:00100000:0.0:1788572564.325357:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1414 00000100:00100000:0.0:1788572564.325367:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1875453382399872:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1788572564.325392:0:6682:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:91 count 1 00002000:00010000:0.0:1788572564.325435:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572564.325443:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.325459:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=912s v=(0 0) 00010000:00010000:0.0:1788572564.325463:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572564.325470:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.325486:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.325497:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.325505:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.325522:0:6682:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00080000:2.0:1788572564.325805:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572564.326231:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382399872:12345-0@lo:6:osp-syn-0-0.0 Request processed in 864us (1056us total) trans 4294967672 rc 0/0 00000100:00100000:0.0:1788572564.326240:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1414 00010000:00010000:0.0:1788572564.326266:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572564.326273:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.326283:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572564.326288:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.326315:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.326333:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.326343:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000689f751f/0x85ace67feb165a81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.326364:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572564.326366:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572564.326368:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1788572564.326527:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627d0fb0a80/2, rc 0, transno 4294967672 00000100:00100000:2.0:1788572564.326535:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382399872:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1788572564.327537:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 42 committed 00000100:00100000:1.0:1788572564.327814:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788572564.327831:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9a5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382400000:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1788572564.327887:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572564.328111:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382400000 00000100:00100000:2.0:1788572564.328119:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382400000 00000100:00100000:2.0:1788572564.328169:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1415 00000100:00100000:2.0:1788572564.328177:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382400000:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1788572564.328200:0:6681:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:90 count 1 00010000:00010000:2.0:1788572564.328223:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.328244:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 82 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.328253:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.328260:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=3s v=(1 0) 00010000:00010000:2.0:1788572564.328265:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(0 1) 00010000:00010000:2.0:1788572564.328270:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.328298:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.328312:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 12 pid: 20911 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.328323:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382400128:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572564.328469:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 82 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.328479:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 82 pid: 6679 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.328489:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382400256:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572564.328514:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.329459:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.329510:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360641024 00000100:00100000:0.0:1788572564.329517:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360641024 00000100:00100000:0.0:1788572564.329574:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1416 00000100:00100000:0.0:1788572564.329584:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360641024:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572564.331765:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382400128:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572564.331780:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7558700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6681:1875453382400256:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572564.331808:0:6681:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.348085:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.348119:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.348188:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360641280 00000100:00100000:1.0:1788572564.348198:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360641280 00010000:00010000:1.0:1788572564.348254:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 83 pid: 6679 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572564.348286:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360641152 00000100:00100000:1.0:1788572564.348288:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360641152 00010000:00010000:1.0:1788572564.348301:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 14 pid: 20911 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572564.348323:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1137 00000100:00100000:1.0:1788572564.348332:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360641280:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572564.348346:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572564.348352:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 83 pid: 6679 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348366:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 83 pid: 6679 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348389:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008ad0899f/0x85ace67feb16582e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f151a expref: 83 pid: 6679 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348406:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572564.348469:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:2350:x1875453360641280:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 137us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.348477:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1137 00000100:00100000:1.0:1788572564.348493:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1138 00000100:00100000:1.0:1788572564.348500:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360641152:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572564.348507:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572564.348512:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 14 pid: 20911 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348521:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 14 pid: 20911 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348533:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c69b4f2e/0x85ace67feb16585f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f152f expref: 14 pid: 20911 timeout: 1012 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348544:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(0 1) 00010000:00010000:1.0:1788572564.348548:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5a:0x0].0x0 since=0s v=(0 1) 00010000:00010000:1.0:1788572564.348554:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348563:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348589:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572564.348608:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360641152:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 109us (489us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.348615:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1138 00010000:00010000:1.0:1788572564.348744:0:6681:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348754:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348766:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348774:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.348791:0:6681:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572564.349324:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3304:x1875453382400000:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21144us (21434us total) trans 4294967671 rc 0/0 00000100:00100000:1.0:1788572564.349332:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1415 00010000:00010000:1.0:1788572564.349355:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572564.349361:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.349370:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572564.349375:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.349385:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.349400:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.349408:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008e8bcae8/0x85ace67feb165a88 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.349427:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572564.349429:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572564.349431:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1788572564.349462:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627c9a5aa00/2, rc 0, transno 4294967671 00000100:00100000:1.0:1788572564.349469:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9a5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382400000:0@lo:6:osp-syn-1-0.0 00000100:00080000:0.0:1788572564.349880:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:1.0:1788572564.359170:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967670 is committed 00000100:00100000:0.0:1788572564.359494:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360641024:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 29836us (29964us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.359508:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1416 00000100:00100000:0.0:1788572564.364478:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.364556:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360641536 00000100:00100000:3.0:1788572564.364565:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360641536 00000100:00100000:3.0:1788572564.364615:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4878 00000100:00100000:3.0:1788572564.364622:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36630:x1875453360641536:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572564.364678:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572564.364701:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb165a6c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15c2 expref: 59 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.364721:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:3.0:1788572564.364726:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572564.364764:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.364779:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.364788:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572564.364987:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006629fa67 reply state 000000000a1f3db6 transno 4294970985 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.364994:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572564.365303:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36630:x1875453360641536:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 681us (836us total) trans 4294970985 rc 0/0 00000100:00100000:3.0:1788572564.365309:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4878 00010000:00010000:0.0:1788572564.369130:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.369147:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.369154:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.369160:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572564.369164:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.369171:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.369187:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:0.0:1788572564.369190:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572564.369194:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.369200:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165a8f lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.370364:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.370506:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360641664 00000100:00100000:1.0:1788572564.370518:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360641664 00000100:00100000:1.0:1788572564.370584:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4879 00000100:00100000:1.0:1788572564.370595:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36630:x1875453360641664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572564.370624:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572564.370649:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15d7 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.370734:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.370750:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.370773:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:1.0:1788572564.370776:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572564.370781:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bdc00, rc 0) 00000100:00100000:1.0:1788572564.370848:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36630:x1875453360641664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 254us (492us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.370860:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4879 00000100:00100000:3.0:1788572564.374846:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.374934:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360641792 00000100:00100000:0.0:1788572564.374947:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360641792 00000100:00100000:0.0:1788572564.375083:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1417 00000100:00100000:0.0:1788572564.375096:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360641792:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572564.375294:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x5b:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1788572564.375297:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x5b:0x0] updating lvb mtime from disk: 0 -> 1788572562 00002000:00010000:0.0:1788572564.375298:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x5b:0x0] updating lvb atime from disk: 0 -> 1788572562 00002000:00010000:0.0:1788572564.375300:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x5b:0x0] updating lvb ctime from disk: 0 -> 1788572562 00002000:00010000:0.0:1788572564.375301:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x5b:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:0.0:1788572564.375348:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360641792:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 257us (516us total) trans 4294967672 rc 0/0 00000100:00100000:0.0:1788572564.375353:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1417 00000100:00100000:2.0:1788572564.378790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.378942:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642048 00000100:00100000:0.0:1788572564.378960:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642048 00000100:00100000:2.0:1788572564.379293:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4880 00000100:00100000:2.0:1788572564.379306:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:36630:x1875453360642048:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572564.379376:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.379393:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 60 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.379404:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 4/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.379442:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 60 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.379455:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 60 pid: 23058 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.379470:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382400384:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572564.379555:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.382772:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572564.382802:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382400384:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572564.382832:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.382922:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642176 00000100:00100000:0.0:1788572564.382930:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642176 00010000:00010000:0.0:1788572564.382987:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 61 pid: 23058 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.383062:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1139 00000100:00100000:2.0:1788572564.383128:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2349:x1875453360642176:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572564.383142:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572564.383147:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 61 pid: 23058 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.383156:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 61 pid: 23058 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.383173:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb165a3b lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa9:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ad expref: 61 pid: 23058 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.383185:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:2.0:1788572564.383189:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.383195:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.383201:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.383222:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572564.383261:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2349:x1875453360642176:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 190us (500us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572564.383269:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1139 00010000:00010000:0.0:1788572564.383320:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383328:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383356:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383364:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383371:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383377:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572564.383382:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383392:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383402:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:0.0:1788572564.383404:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572564.383409:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.383415:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb165a9d lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.383459:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36630:x1875453360642048:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 4156us (4677us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572564.383467:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4880 00000100:00100000:3.0:1788572564.386827:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.386926:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642304 00000100:00100000:1.0:1788572564.386935:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642304 00000100:00100000:1.0:1788572564.386995:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1418 00000100:00100000:1.0:1788572564.387211:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360642304:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572564.387256:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.387277:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.387286:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572564.387290:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572564.387295:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.387330:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.387343:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 11 pid: 13272 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572564.387358:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382400512:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572564.387464:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c11ca3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572564.396494:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382400512:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572564.396531:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.402840:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.407040:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642432 00000100:00100000:1.0:1788572564.407056:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642432 00010000:00010000:1.0:1788572564.407107:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 12 pid: 13272 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572564.407223:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1140 00000100:00100000:1.0:1788572564.407234:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360642432:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572564.407250:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572564.407274:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 12 pid: 13272 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.407289:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 12 pid: 13272 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.407312:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb165a7a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d0 expref: 12 pid: 13272 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.407330:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572564.407335:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572564.407341:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.407352:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.407386:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572564.407454:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360642432:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 222us (4623us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.407462:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1140 00010000:00010000:0.0:1788572564.407699:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407710:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407736:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407743:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407751:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407758:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572564.407762:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407772:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407784:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.407792:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000e37eb3a/0x85ace67feb165aa4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.407858:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360642304:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 20854us (21042us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.407866:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1418 00000100:00100000:3.0:1788572564.412306:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.412518:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642688 00000100:00100000:0.0:1788572564.412528:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642688 00000100:00100000:0.0:1788572564.412637:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4881 00000100:00100000:0.0:1788572564.412660:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:36630:x1875453360642688:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572564.412739:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.412757:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.412770:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.412986:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.412998:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.413044:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.413078:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.413093:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 60 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.413108:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382400640:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572564.413187:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982da80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572564.417382:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.417625:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642816 00000100:00100000:3.0:1788572564.417634:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642816 00010000:00010000:3.0:1788572564.417672:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 61 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.417719:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1141 00000100:00100000:3.0:1788572564.417729:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360642816:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572564.417743:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572564.417750:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 61 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.417764:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 61 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.417785:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:3.0:1788572564.417792:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.417802:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.417810:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.417828:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572564.418205:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360642816:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 475us (1026us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572564.418214:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1141 00010000:00010000:0.0:1788572564.418408:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165a96 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15d7 expref: 60 pid: 5837 timeout: 1012 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572564.418425:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382400640:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572564.418458:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.418468:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.418474:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572564.418673:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970986 00000001:00080000:0.0:1788572564.418810:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970987 00000001:00080000:0.0:1788572564.418856:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970988 00000100:00100000:0.0:1788572564.418937:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382400768:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1788572564.418993:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572564.419048:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c8550e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.419233:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382400768 00000100:00100000:1.0:1788572564.419238:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382400768 00000100:00100000:1.0:1788572564.420346:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:1.0:1788572564.420359:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382400768:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572564.420456:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382400768:12345-0@lo:1000:mdt00_001.0 Request processed in 100us (1462us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572564.420464:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1788572564.420559:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382400768:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1788572564.421148:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970989 00000100:00100000:3.0:1788572564.421217:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382400896:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1788572564.421256:0:5838:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572564.421277:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982da80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572564.421411:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382400896 00000100:00100000:1.0:1788572564.421417:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382400896 00000100:00100000:1.0:1788572564.421459:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1788572564.421468:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5838:x1875453382400896:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1788572564.421638:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5838:x1875453382400896:12345-0@lo:1000:mdt00_001.0 Request processed in 171us (381us total) trans 4294967673 rc 0/0 00000100:00100000:1.0:1788572564.421658:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1788572564.421884:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5838:1875453382400896:0@lo:1000:mdt00_001.0 00000001:00080000:2.0:1788572564.421978:0:5838:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970990 00010000:00010000:2.0:1788572564.422038:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422048:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422056:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422061:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572564.422066:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422074:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422090:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:2.0:1788572564.422093:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572564.422097:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422104:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb165ab2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422132:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422143:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422150:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422155:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572564.422158:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422164:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422171:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627c5160400) 00010000:00010000:2.0:1788572564.422174:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572564.422181:0:5838:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 239 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422193:0:5838:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 239 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422202:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 239 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572564.422232:0:5838:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647000/0x85ace67feb1659e0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1598 expref: 239 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572564.422244:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382401024:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572564.422355:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788572564.425817:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.425937:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382401024:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572564.425978:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.425988:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1659d9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426056:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426069:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426084:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426096:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426102:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426108:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426114:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572564.426118:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426125:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426136:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627c5160400) 00010000:00010000:3.0:1788572564.426139:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572564.426143:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426148:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5600/0x85ace67feb165ab9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xa8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426158:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426163:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426169:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426174:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572564.426176:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426182:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426188:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa8:0x0].0x0 (ffff9627c5160400) 00010000:00010000:3.0:1788572564.426191:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572564.426194:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 2/0,0 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.426199:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165aab lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572564.426444:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360642688:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 13796us (14147us total) trans 4294970991 rc 0/0 00000100:00100000:3.0:1788572564.426454:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4881 00000020:00080000:3.0:1788572564.426459:0:5838:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:0.0:1788572564.426534:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360642944 00000100:00100000:0.0:1788572564.426541:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360642944 00000100:00100000:0.0:1788572564.426592:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1058 00000100:00100000:0.0:1788572564.426603:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:36628:x1875453360642944:12345-192.168.202.40@tcp:49:md5sum.0 00000100:00100000:0.0:1788572564.426713:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:36628:x1875453360642944:12345-192.168.202.40@tcp:49:md5sum.0 Request processed in 112us (907us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572564.426720:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1058 00000040:00080000:2.0:1788572564.427112:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572564.427117:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572564.427709:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1788572564.427718:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572564.431517:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.431597:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360643200 00000100:00100000:0.0:1788572564.431608:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360643200 00000100:00100000:0.0:1788572564.431691:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1059 00000100:00100000:0.0:1788572564.431702:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:36630:x1875453360643200:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572564.432411:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xa9:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572564.432549:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294970992 00010000:00010000:0.0:1788572564.432599:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.432615:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.432626:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.432634:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.432644:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.432650:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572564.432677:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572564.432772:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:36630:x1875453360643200:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1071us (1264us total) trans 4294970993 rc 0/0 00000100:00100000:0.0:1788572564.432780:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1059 00000020:00080000:0.0:1788572564.432785:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572564.432917:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572564.432924:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.432931:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572564.432935:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.432944:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.432967:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa9:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:3.0:1788572564.432970:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572564.432975:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 1/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.432980:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ac0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xa9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.432997:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572564.432999:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572564.433001:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572564.435811:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.435919:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360643328 00000100:00100000:3.0:1788572564.435929:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360643328 00000100:00100000:3.0:1788572564.435995:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1419 00000100:00100000:3.0:1788572564.436038:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+82:36628:x1875453360643328:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572564.436056:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572564.436076:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008b550ad7/0x85ace67feb165ac7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15de expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572564.436104:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5b:0x0] initial LVB size: 1048576, mtime: 0x6a9b7392, atime: 0x6a9b7392, ctime: 0x6a9b7392, blocks: 0x803 00010000:00010000:3.0:1788572564.436127:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=912s v=(0 0) 00010000:00010000:3.0:1788572564.436132:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572564.436141:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008b550ad7/0x85ace67feb165ac7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15de expref: 83 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.436162:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008b550ad7/0x85ace67feb165ac7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15de expref: 83 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572564.436174:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5600, rc 0) 00000100:00100000:3.0:1788572564.436290:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:36628:x1875453360643328:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 286us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572564.436297:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1419 00000100:00100000:1.0:1788572564.440875:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572564.440981:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360643456 00000100:00100000:0.0:1788572564.440992:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360643456 00002000:00100000:0.0:1788572564.441084:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5b:0x0] (0->32767) req@ffff9626c5431500 x1875453360643456/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:225/0 lens 488/0 e 0 to 0 dl 1788572575 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572564.441096:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431500 with x1875453360643456 ext(0->32767) 00002000:00010000:0.0:1788572564.441103:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5431500 00000100:00100000:0.0:1788572564.441127:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 360 00000100:00100000:0.0:1788572564.441138:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2347:x1875453360643456:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:0.0:1788572564.447740:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2347:x1875453360643456:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 6593us (6864us total) trans 0 rc 32768/32768 00000100:00100000:0.0:1788572564.447756:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 360 00002000:00100000:0.0:1788572564.447770:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5b:0x0] (0->32767) req@ffff9626c5431500 x1875453360643456/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:225/0 lens 488/440 e 0 to 0 dl 1788572575 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572564.447781:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431500 with x1875453360643456 ext(0->32767) 00002000:00010000:0.0:1788572564.447788:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5431500 00000100:00100000:1.0:1788572564.457584:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.457764:0:6685:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360643584 00000100:00100000:3.0:1788572564.457776:0:6685:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360643584 00002000:00100000:3.0:1788572564.457825:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5b:0x0] (32768->1048575) req@ffff9626c5430000 x1875453360643584/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:225/0 lens 488/0 e 0 to 0 dl 1788572575 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572564.457833:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5430000 with x1875453360643584 ext(32768->1048575) 00002000:00010000:3.0:1788572564.457838:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5430000 00000100:00100000:3.0:1788572564.457875:0:6685:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 361 00000100:00100000:3.0:1788572564.457885:0:6685:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2347:x1875453360643584:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572564.474158:0:6685:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2347:x1875453360643584:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 16259us (16570us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572564.474176:0:6685:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 361 00002000:00100000:2.0:1788572564.474191:0:6685:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5b:0x0] (32768->1048575) req@ffff9626c5430000 x1875453360643584/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:225/0 lens 488/440 e 0 to 0 dl 1788572575 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572564.474204:0:6685:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5430000 with x1875453360643584 ext(32768->1048575) 00002000:00010000:2.0:1788572564.474212:0:6685:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c5430000 00000100:00100000:1.0:1788572564.484405:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572564.484556:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360643840 00000100:00100000:3.0:1788572564.484565:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360643840 00000100:00100000:3.0:1788572564.484627:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1060 00000100:00100000:3.0:1788572564.484637:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:36628:x1875453360643840:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:3.0:1788572564.484910:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:36628:x1875453360643840:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 273us (514us total) trans 4294970994 rc 0/0 00000100:00100000:3.0:1788572564.484918:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1060 00000100:00100000:3.0:1788572564.599491:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572564.599616:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360643968 00000100:00100000:1.0:1788572564.599627:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360643968 00000100:00100000:1.0:1788572564.599687:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 397 00000100:00100000:1.0:1788572564.599707:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2348:x1875453360643968:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572564.599747:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572564.599767:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ace lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15e5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.599831:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.599846:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.599859:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.599874:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.599881:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.599886:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600164:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600173:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600180:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600185:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572564.600190:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600202:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600215:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572564.600218:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572564.600223:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600230:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165ad5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600244:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600249:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600255:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600259:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572564.600262:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600269:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600275:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xef59b368 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572564.600277:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572564.600280:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600289:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165adc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xef59b368 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600317:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ace lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15e5 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600328:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572564.600330:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572564.600334:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ace lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15e5 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572564.600342:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc -2) 00000100:00100000:1.0:1788572564.600615:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2348:x1875453360643968:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 918us (1134us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572564.600624:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 397 00010000:00010000:1.0:1788572564.602364:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572565.103274:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572565.103743:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644096 00000100:00100000:0.0:1788572565.103757:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644096 00000100:00100000:0.0:1788572565.103846:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 398 00000100:00100000:0.0:1788572565.103859:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2348:x1875453360644096:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572565.103897:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572565.103918:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ae3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15ec expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104031:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104050:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104066:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104083:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104091:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104097:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104313:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104322:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104330:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104336:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572565.104340:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104348:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104361:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572565.104365:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572565.104369:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104376:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb165aea lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104387:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104393:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104412:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104416:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572565.104419:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104436:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104444:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x2425dbd8 (ffff9627c5160d00) 00010000:00010000:0.0:1788572565.104446:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572565.104450:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104455:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165af1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x2425dbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104486:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ae3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ec expref: 67 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104495:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:0.0:1788572565.104498:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572565.104502:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ae3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15ec expref: 67 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572565.104511:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646000, rc -2) 00000100:00100000:0.0:1788572565.104592:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2348:x1875453360644096:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 736us (1329us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572565.104601:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 398 00000100:00100000:2.0:1788572565.116666:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572565.116781:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644224 00000100:00100000:1.0:1788572565.116793:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644224 00000100:00100000:1.0:1788572565.116858:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 399 00000100:00100000:1.0:1788572565.116869:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360644224:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572565.116911:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572565.116945:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165af8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15f3 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117053:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117068:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117080:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117094:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117102:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117109:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117314:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117322:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117328:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117334:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572565.117338:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117345:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117359:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572565.117363:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572565.117368:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117375:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165aff lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117386:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117391:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117409:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117414:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572565.117417:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117644:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117653:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xa99f4bc1 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572565.117655:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572565.117659:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117664:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b06 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xa99f4bc1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117696:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165af8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15f3 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117707:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572565.117709:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572565.117713:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165af8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15f3 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.117722:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc -2) 00000100:00100000:1.0:1788572565.117853:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360644224:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 985us (1196us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572565.117861:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 399 00000100:00100000:2.0:1788572565.619946:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572565.620098:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644352 00000100:00100000:1.0:1788572565.620108:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644352 00000100:00100000:1.0:1788572565.620167:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 400 00000100:00100000:1.0:1788572565.620177:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360644352:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572565.620217:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572565.620240:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b0d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f15fa expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620306:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620323:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620335:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620350:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620358:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620365:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620579:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620589:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620595:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620601:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572565.620606:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620614:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620626:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572565.620628:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572565.620633:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620639:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165b14 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620651:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620657:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620662:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620668:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572565.620671:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620676:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620683:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xc71d5e4c (ffff9627ff5abd00) 00010000:00010000:1.0:1788572565.620686:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572565.620689:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620695:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b1b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xc71d5e4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620723:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b0d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15fa expref: 67 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620732:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572565.620734:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572565.620738:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b0d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f15fa expref: 67 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.620753:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc -2) 00000100:00100000:1.0:1788572565.620815:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360644352:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 639us (878us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572565.620822:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 400 00010000:00010000:0.0:1788572565.626213:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572565.644337:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572565.644481:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644480 00000100:00100000:1.0:1788572565.644494:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644480 00000100:00100000:1.0:1788572565.644567:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 401 00000100:00100000:1.0:1788572565.644578:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360644480:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572565.644645:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572565.644685:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b22 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1601 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.644754:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.644771:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.644784:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.644800:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.644807:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.644814:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645216:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645229:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645235:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645242:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572565.645247:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645255:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645269:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572565.645272:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572565.645277:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645285:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b29 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645297:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645303:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645310:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645316:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572565.645319:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645325:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645332:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0xcbce33d8 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572565.645334:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572565.645337:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645343:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b30 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0xcbce33d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645458:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b22 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1601 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645471:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:1.0:1788572565.645474:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572565.645478:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b22 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1601 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572565.645488:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc000, rc -2) 00000100:00100000:1.0:1788572565.645616:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360644480:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1040us (1297us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572565.645625:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 401 00000100:00100000:0.0:1788572566.140436:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140483:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140512:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140531:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140551:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140569:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140592:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572566.140621:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644864 00000100:00100000:0.0:1788572566.140630:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644864 00000100:00100000:0.0:1788572566.140721:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645376 00000100:00100000:0.0:1788572566.140725:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645376 00000100:00100000:0.0:1788572566.140743:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644992 00000100:00100000:0.0:1788572566.140745:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644992 00000100:00100000:0.0:1788572566.140758:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645248 00000100:00100000:0.0:1788572566.140760:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645248 00000100:00100000:0.0:1788572566.140775:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 616 00000100:00100000:0.0:1788572566.140784:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2349:x1875453360644864:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572566.141242:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644608 00000100:00100000:3.0:1788572566.141248:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644608 00000100:00100000:3.0:1788572566.141280:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 687 00000100:00100000:3.0:1788572566.141287:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2349:x1875453360644608:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572566.141344:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2349:x1875453360644608:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 57us (918us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572566.141362:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 687 00000100:00100000:0.0:1788572566.141705:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2349:x1875453360644864:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 920us (1219us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572566.141713:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 616 00000100:00100000:0.0:1788572566.141730:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 617 00000100:00100000:0.0:1788572566.141738:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360645376:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572566.141770:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360645376:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 33us (1237us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572566.141776:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 617 00000100:00100000:0.0:1788572566.141787:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 618 00000100:00100000:0.0:1788572566.141794:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2349:x1875453360644992:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572566.141815:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2349:x1875453360644992:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 22us (1245us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572566.141835:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 618 00000100:00100000:0.0:1788572566.141847:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 619 00000100:00100000:0.0:1788572566.141854:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb33fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2349:x1875453360645248:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572566.141894:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb33fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2349:x1875453360645248:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 31us (1290us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572566.141900:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 619 00000100:00100000:0.0:1788572566.142061:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645120 00000100:00100000:0.0:1788572566.142066:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645120 00000100:00100000:0.0:1788572566.142228:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360644736 00000100:00100000:0.0:1788572566.142232:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360644736 00000100:00100000:0.0:1788572566.142253:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 647 00000100:00100000:0.0:1788572566.142260:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360645120:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572566.142292:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2349:x1875453360645120:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 33us (1780us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572566.142298:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 647 00000100:00100000:0.0:1788572566.142309:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 648 00000100:00100000:0.0:1788572566.142314:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2349:x1875453360644736:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572566.142491:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2349:x1875453360644736:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 175us (1938us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572566.142498:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 648 00000100:00080000:0.0:1788572566.142530:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1788572566.142534:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788572566.142536:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1788572566.650975:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572567.666896:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572567.666981:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645504 00000100:00100000:2.0:1788572567.666993:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645504 00000100:00100000:2.0:1788572567.667078:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 402 00000100:00100000:2.0:1788572567.667088:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2350:x1875453360645504:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572567.667126:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572567.667147:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165b37 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1608 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572567.667208:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 3/1,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572567.667227:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572567.667240:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572567.667254:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572567.667262:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572567.667268:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671491:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671511:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671520:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671526:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572567.671531:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671539:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671557:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:3.0:1788572567.671560:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572567.671566:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671572:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b3e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671584:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671591:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671597:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671602:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572567.671604:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671610:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671617:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x8799b2a3 (ffff9627c613b800) 00010000:00010000:3.0:1788572567.671620:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572567.671623:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671629:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b45 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671682:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165b37 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1608 expref: 242 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671693:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffff9627d55e6000) 00010000:00010000:3.0:1788572567.671695:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572567.671700:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165b37 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1608 expref: 242 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572567.671709:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8c00, rc -2) 00000100:00100000:3.0:1788572567.671859:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2350:x1875453360645504:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 4770us (4972us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572567.671870:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 402 00010000:00010000:0.0:1788572567.675159:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572567.876156:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382401152:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788572567.876234:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572567.876387:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401152 00000100:00100000:2.0:1788572567.876398:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401152 00000100:00100000:2.0:1788572567.876509:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:2.0:1788572567.876519:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382401152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788572567.876555:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 192086016 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1788572567.876608:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382401152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572567.876615:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:2.0:1788572567.876661:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382401152:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1788572567.888983:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572568.384201:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788572568.384231:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3305:1875453382401408:0@lo:400:kworker.0 00000100:00100000:3.0:1788572568.384324:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572568.384386:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401408 00000100:00100000:3.0:1788572568.384394:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401408 00000100:00100000:3.0:1788572568.384443:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:3.0:1788572568.384452:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1875453382401408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572568.384504:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1875453382401408:12345-0@lo:400:kworker.0 Request processed in 53us (180us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572568.384510:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:3.0:1788572568.384550:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3305:1875453382401408:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.384706:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788572568.384718:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3302:1875453382401280:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.384765:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572568.384817:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401280 00000100:00100000:1.0:1788572568.384821:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401280 00000100:00100000:1.0:1788572568.384848:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 688 00000100:00100000:1.0:1788572568.384855:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0eff450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3302:x1875453382401280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.384896:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0eff450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3302:x1875453382401280:12345-0@lo:400:kworker.0 Request processed in 41us (129us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572568.384902:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 688 00000100:00100000:1.0:1788572568.384933:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3302:1875453382401280:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.385222:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3305:1875453382401536:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.385250:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572568.385278:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788572568.385283:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382401920:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.385319:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572568.385342:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382401664:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.385362:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572568.385376:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788572568.385381:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382401792:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.385399:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572568.386116:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401920 00000100:00100000:0.0:1788572568.386125:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401920 00000100:00100000:0.0:1788572568.386188:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401664 00000100:00100000:0.0:1788572568.386191:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401664 00000100:00100000:0.0:1788572568.386230:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:0.0:1788572568.386238:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382401920:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1788572568.386380:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788572568.386631:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382401920:12345-0@lo:400:kworker.0 Request processed in 393us (1309us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572568.386639:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:0.0:1788572568.386656:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:0.0:1788572568.386665:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382401664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788572568.386707:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3302:x1875453382401664:12345-0@lo:400:kworker.0 Request processed in 46us (1343us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572568.386713:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:0.0:1788572568.386822:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401536 00000100:00100000:0.0:1788572568.386827:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401536 00000100:00100000:0.0:1788572568.386860:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382401792 00000100:00100000:0.0:1788572568.386863:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382401792 00000100:00100000:0.0:1788572568.386889:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1788572568.386897:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1875453382401536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788572568.386931:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1875453382401536:12345-0@lo:400:kworker.0 Request processed in 36us (1681us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572568.386938:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1788572568.386950:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:0.0:1788572568.386958:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382401792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788572568.386986:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1875453382401792:12345-0@lo:400:kworker.0 Request processed in 30us (1585us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572568.386991:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00080000:0.0:1788572568.387042:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572568.387078:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382401920:0@lo:400:kworker.0 00000100:00100000:0.0:1788572568.387092:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3305:1875453382401536:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.387179:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3302:1875453382401792:0@lo:400:kworker.0 00000100:00100000:1.0:1788572568.387191:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382401664:0@lo:400:kworker.0 00000100:00100000:3.0:1788572568.693176:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572568.693671:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645632 00000100:00100000:1.0:1788572568.693691:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645632 00000100:00100000:1.0:1788572568.693836:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 403 00000100:00100000:1.0:1788572568.693851:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2350:x1875453360645632:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572568.693904:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572568.693945:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b4c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f160f expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.694106:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.694126:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788572568.694793:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788572568.696538:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.696596:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.696612:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.696623:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572568.698826:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572568.700211:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700231:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700240:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700247:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572568.700252:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700268:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700348:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0x0 (ffff9627d20cbd00) 00010000:00010000:1.0:1788572568.700352:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572568.700358:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700365:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b53 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700387:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700393:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700407:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700534:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572568.700538:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700545:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700559:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0x8799b2a3 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572568.700562:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572568.700565:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700571:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b5a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x60:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700627:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b4c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f160f expref: 242 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700638:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x60:0x0].0x0 (ffff9627d20cbd00) 00010000:00010000:1.0:1788572568.700640:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572568.700644:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b4c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x60:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f160f expref: 242 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572568.700654:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc000, rc -2) 00000100:00100000:0.0:1788572568.701210:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:2350:x1875453360645632:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 7356us (8037us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572568.701224:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 403 00000100:00100000:2.0:1788572568.890640:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1788572568.890674:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382402048:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788572568.890790:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572568.890976:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382402048 00000100:00100000:1.0:1788572568.890986:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382402048 00000100:00100000:1.0:1788572568.891091:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:1.0:1788572568.891102:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3304:x1875453382402048:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788572568.891156:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 195493888 pending 0 free 3854565376 avail 3852468224 00000100:00100000:1.0:1788572568.891216:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3304:x1875453382402048:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572568.891224:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1788572568.891612:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382402048:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1788572569.038278:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967672 is committed 00000001:00080000:1.0:1788572569.503237:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967672 is committed 00000001:00080000:1.0:1788572569.503260:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967673 is committed 00000001:00080000:1.0:1788572569.520525:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970993 is committed 00000001:00080000:1.0:1788572569.520564:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970994 is committed 00000020:00080000:1.0:1788572569.520577:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2f566c0 refcount 2 committed 0 result 0 batchid 146 00000020:00080000:1.0:1788572569.520583:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79aeb00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c2f56840 00000020:00080000:1.0:1788572569.520587:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627e79ae200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ff03c800 00000020:00080000:2.0:1788572569.520979:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 146 committed batchid 145 00000020:00080000:2.0:1788572569.521272:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 145 00000040:00080000:1.0:1788572569.530349:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572569.530363:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572569.530416:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 169 count 3 00000040:00080000:1.0:1788572569.530425:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1788572569.723262:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572570.579305:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572570.579385:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645760 00000100:00100000:3.0:1788572570.579396:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645760 00000100:00100000:3.0:1788572570.579474:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4882 00000100:00100000:3.0:1788572570.579485:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36573:x1875453360645760:12345-192.168.202.40@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1788572570.579599:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:36573:x1875453360645760:12345-192.168.202.40@tcp:60:lhsmtool_posix.0 Request processed in 115us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572570.579607:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4882 00000100:00080000:0.0:1788572570.579764:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788572570.746324:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572571.258985:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572571.259125:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572571.259169:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572571.259220:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646144 00000100:00100000:2.0:1788572571.259223:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572571.259240:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646144 00000100:00100000:2.0:1788572571.259253:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572571.259283:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572571.259346:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646656 00000100:00100000:2.0:1788572571.259352:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572571.259354:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646656 00000100:00100000:0.0:1788572571.259402:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646272 00000100:00100000:2.0:1788572571.259404:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646400 00000100:00100000:0.0:1788572571.259407:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646272 00000100:00100000:2.0:1788572571.259415:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646400 00000100:00100000:0.0:1788572571.259433:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646528 00000100:00100000:0.0:1788572571.259438:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646528 00000100:00100000:0.0:1788572571.259482:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 622 00000100:00100000:2.0:1788572571.259486:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646016 00000100:00100000:2.0:1788572571.259492:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646016 00000100:00100000:0.0:1788572571.259500:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2350:x1875453360646144:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572571.259531:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 652 00000100:00100000:2.0:1788572571.259543:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2350:x1875453360646400:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572571.259621:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2350:x1875453360646144:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 125us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572571.259636:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 622 00000100:00100000:2.0:1788572571.259638:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2350:x1875453360646400:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 93us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572571.259653:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 652 00000100:00100000:0.0:1788572571.259681:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 623 00000100:00100000:2.0:1788572571.259699:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 653 00000100:00100000:0.0:1788572571.259707:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360646656:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572571.259718:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:2350:x1875453360646016:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572571.259758:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:2350:x1875453360646016:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 48us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572571.259765:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 653 00000100:00100000:0.0:1788572571.259780:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360646656:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 76us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572571.259791:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 623 00000100:00100000:2.0:1788572571.259801:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 624 00000100:00100000:2.0:1788572571.259814:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360646272:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:0.0:1788572571.259822:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 625 00000100:00100000:0.0:1788572571.259844:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360646528:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572571.259873:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360646272:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 61us (588us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572571.259882:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 624 00000100:00100000:0.0:1788572571.259900:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360646528:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 59us (548us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572571.259909:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 625 00000100:00080000:2.0:1788572571.260256:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788572571.260264:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788572571.260267:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572571.260336:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360645888 00000100:00100000:3.0:1788572571.260341:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360645888 00000100:00100000:3.0:1788572571.260386:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 689 00000100:00100000:3.0:1788572571.260395:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2350:x1875453360645888:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572571.260437:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2350:x1875453360645888:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 44us (1469us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572571.260444:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 689 00010000:00010000:0.0:1788572571.774842:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788572572.218074:0:23748:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 15: Rebind a list of files ============ 21:42:50 (1788572570) 00000100:00100000:2.0:1788572572.277177:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572572.277294:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646784 00000100:00100000:0.0:1788572572.277313:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646784 00000100:00100000:0.0:1788572572.277414:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 404 00000100:00100000:0.0:1788572572.277429:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:2350:x1875453360646784:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572572.277486:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572572.277520:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165b61 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1616 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.277612:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.277632:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.277662:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.277692:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.277700:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.277707:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278122:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278255:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278263:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278269:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.278275:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278284:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278300:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572572.278303:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.278310:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278317:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165b68 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278332:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278338:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278344:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278351:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.278354:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278361:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278367:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x8799b2a3 (ffff9627c5160900) 00010000:00010000:0.0:1788572572.278369:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.278371:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278376:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165b6f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278420:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165b61 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1616 expref: 241 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278432:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffff9627d55e6700) 00010000:00010000:0.0:1788572572.278435:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572572.278439:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb165b61 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1616 expref: 241 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.278449:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647600, rc -2) 00000100:00100000:0.0:1788572572.278544:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:2350:x1875453360646784:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1118us (1377us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572572.278554:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 404 00000100:00100000:3.0:1788572572.656129:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572572.656534:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360646912 00000100:00100000:2.0:1788572572.656549:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360646912 00000100:00100000:2.0:1788572572.656728:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4883 00000100:00100000:2.0:1788572572.656744:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:37238:x1875453360646912:12345-192.168.202.40@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1788572572.656900:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:37238:x1875453360646912:12345-192.168.202.40@tcp:59:lhsmtool_posix.0 Request processed in 160us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572572.656908:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4883 00000100:00080000:0.0:1788572572.658838:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572572.678264:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572572.678363:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647040 00000100:00100000:2.0:1788572572.678376:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647040 00000100:00100000:2.0:1788572572.678625:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4884 00000100:00100000:2.0:1788572572.678648:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:37239:x1875453360647040:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572572.678678:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572572.678708:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165b76 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f161d expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.678801:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.678819:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.678836:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.678854:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.678862:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.678869:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679600:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679614:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679622:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679638:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572572.679643:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679652:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679669:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:2.0:1788572572.679672:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572572.679679:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679686:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165b7d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679701:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679707:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679713:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679718:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572572.679721:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679727:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679735:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54976cc (ffff9627c613b800) 00010000:00010000:2.0:1788572572.679737:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572572.679740:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679746:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165b84 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679794:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165b76 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f161d expref: 241 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679813:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:2.0:1788572572.679815:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572572.679820:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165b76 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f161d expref: 241 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.679829:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9c00, rc 0) 00000100:00100000:2.0:1788572572.680276:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:37239:x1875453360647040:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1627us (2026us total) trans 0 rc 301/301 00000100:00100000:2.0:1788572572.680285:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4884 00000100:00100000:3.0:1788572572.685221:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572572.690620:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647296 00000100:00100000:1.0:1788572572.690654:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647296 00000100:00100000:1.0:1788572572.690759:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4885 00000100:00100000:1.0:1788572572.690773:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:37239:x1875453360647296:12345-192.168.202.40@tcp:36:lfs.0 00010000:00010000:1.0:1788572572.690855:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572572.690892:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1657b7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14f0 expref: 240 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.690920:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:1.0:1788572572.690938:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572572.692875:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.692914:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 56 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.692943:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.693037:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 56 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.693060:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 56 pid: 22536 timeout: 1020 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572572.693079:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36c4380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382402176:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572572.694738:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572572.703824:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572572.704159:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647424 00000100:00100000:0.0:1788572572.704172:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647424 00010000:00010000:0.0:1788572572.704234:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 57 pid: 22536 timeout: 1020 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572572.704295:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1142 00000100:00100000:0.0:1788572572.704307:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2350:x1875453360647424:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572572.704329:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572572.704338:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 57 pid: 22536 timeout: 1020 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.704354:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 57 pid: 22536 timeout: 1020 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.704393:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:0.0:1788572572.704400:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.704412:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.704421:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.704447:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572572.704527:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:2350:x1875453360647424:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 221us (717us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572572.704534:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1142 00010000:00010000:1.0:1788572572.704886:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16571d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f14bf expref: 56 pid: 22536 timeout: 1020 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572572.704905:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36c4380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382402176:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572572.704947:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.704958:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.704964:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.704993:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.705033:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.705043:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572572.712670:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00010000:00010000:3.0:1788572572.712698:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.712711:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.712719:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.712726:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572572.712732:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.712742:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.712764:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c5160a00) 00010000:00010000:3.0:1788572572.712768:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572572.712774:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.712781:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165b8b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572572.712806:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000773e8997 reply state 00000000f1fc1357 transno 4294970995 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572572.712817:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572572.713350:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:37239:x1875453360647296:12345-192.168.202.40@tcp:36:lfs.0 Request processed in 22571us (28134us total) trans 4294970995 rc 0/0 00000100:00100000:3.0:1788572572.713361:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4885 00010000:00010000:0.0:1788572572.721878:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.721905:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.721913:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.721919:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.721925:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.721934:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.721955:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54976cc (ffff9627ff5ab400) 00010000:00010000:0.0:1788572572.721959:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.721965:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.721973:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165b92 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572572.784269:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572572.784331:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572572.784369:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572572.784405:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572572.784508:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647680 00000100:00100000:0.0:1788572572.784522:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647680 00000100:00100000:1.0:1788572572.784585:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647936 00000100:00100000:1.0:1788572572.784592:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647936 00000100:00100000:0.0:1788572572.784593:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647808 00000100:00100000:0.0:1788572572.784599:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647808 00000100:00100000:1.0:1788572572.784635:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 405 00000100:00100000:1.0:1788572572.784649:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:2349:x1875453360647680:12345-192.168.202.40@tcp:63:ll_sa.0 00000100:00100000:2.0:1788572572.784691:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360647552 00000100:00100000:2.0:1788572572.784696:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360647552 00000100:00100000:2.0:1788572572.784745:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4886 00000100:00100000:2.0:1788572572.784754:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:37246:x1875453360647552:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:2.0:1788572572.784768:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572572.784769:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572572.784798:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b99 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f162b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.784798:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165ba0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1624 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.784887:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ba7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.784905:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.784919:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.784951:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.784963:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572572.784965:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788572572.784974:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00010000:00010000:1.0:1788572572.784977:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.784996:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572572.785038:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.785040:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.785059:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.785080:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165ba0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1624 expref: 62 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.785104:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ba7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1624 expref: 61 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.785114:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c613b800) 00010000:00010000:2.0:1788572572.785117:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572572.785122:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8c00, rc 0) 00000100:00100000:2.0:1788572572.785202:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37246:x1875453360647552:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 449us (948us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572572.785210:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4886 00000100:00100000:0.0:1788572572.788353:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 406 00000100:00100000:0.0:1788572572.788381:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2348:x1875453360647936:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572572.788433:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572572.788450:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.788457:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165bbc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1639 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788463:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788476:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788483:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bca00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572572.788489:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788499:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788519:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572572.788523:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.788528:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788529:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788548:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bca00/0x85ace67feb165bae lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.788556:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788565:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.788570:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788574:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788580:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788587:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572572.788590:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788597:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788605:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x7f9846d8 (ffff9627ff5ab600) 00010000:00010000:2.0:1788572572.788608:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572572.788611:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788617:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb165bb5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x7f9846d8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788773:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b99 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f162b expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788786:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572572.788790:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572572.788794:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc200/0x85ace67feb165b99 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f162b expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.788805:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc200, rc -2) 00000100:00100000:2.0:1788572572.788979:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:2349:x1875453360647680:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 4333us (4646us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572572.788991:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 405 00000100:00100000:2.0:1788572572.789059:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 407 00000100:00100000:2.0:1788572572.789071:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:2350:x1875453360647808:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572572.789092:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572572.789102:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165bca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1632 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789129:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789138:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789145:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789165:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789172:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789178:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.789275:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.789284:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.789294:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789323:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789332:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789338:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789344:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572572.789348:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.789356:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.789358:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.789371:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.789377:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572572.790513:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572572.790551:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360648192 00000100:00100000:1.0:1788572572.790563:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360648192 00000100:00100000:1.0:1788572572.790621:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4887 00000100:00100000:1.0:1788572572.790641:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37246:x1875453360648192:12345-192.168.202.40@tcp:36:mkdir.0 00010000:00010000:1.0:1788572572.790711:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572572.790737:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ba7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1624 expref: 61 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.790759:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627c613b800) 00010000:00010000:1.0:1788572572.790773:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572572.790942:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37246:x1875453360648192:12345-192.168.202.40@tcp:36:mkdir.0 Request processed in 302us (434us total) trans 0 rc -17/-17 00000100:00100000:1.0:1788572572.790950:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4887 00010000:00010000:2.0:1788572572.791161:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572572.791166:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572572.791173:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791182:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165bd1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791199:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791206:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791213:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791219:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572572.791223:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791233:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791240:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x558c17a (ffff9627d20cb300) 00010000:00010000:2.0:1788572572.791243:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572572.791246:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791252:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165bd8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x558c17a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791285:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165bca lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1632 expref: 59 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791294:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:2.0:1788572572.791297:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572572.791301:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165bca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1632 expref: 59 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.791311:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8200, rc -2) 00000100:00100000:2.0:1788572572.791362:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:2350:x1875453360647808:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2294us (6956us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572572.791371:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 407 00000100:00100000:2.0:1788572572.793784:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572572.793854:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360648320 00000100:00100000:1.0:1788572572.793868:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360648320 00000100:00100000:1.0:1788572572.793945:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4888 00000100:00100000:1.0:1788572572.793957:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:37246:x1875453360648320:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:1.0:1788572572.793983:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572572.794033:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165be6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1640 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.794124:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165bed lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.794141:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165bed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.794156:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165bed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572572.794214:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1788572572.794218:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00010000:00010000:0.0:1788572572.794677:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.794687:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794703:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794721:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572572.794724:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.794730:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794738:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165bc3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794758:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794764:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794771:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794776:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.794779:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794785:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794792:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0xfb4f31d (ffff9627c5160900) 00010000:00010000:0.0:1788572572.794794:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.794798:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794808:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165bdf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0xfb4f31d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794849:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165bbc lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1639 expref: 59 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794860:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:0.0:1788572572.794862:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572572.794867:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb165bbc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1639 expref: 59 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.794877:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647400, rc -2) 00000004:00010000:1.0:1788572572.795179:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165bed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.795203:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165be6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1640 expref: 59 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.795220:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc000/0x85ace67feb165bed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1640 expref: 58 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572572.795228:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627ff5ab200) 00010000:00010000:1.0:1788572572.795230:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572572.795309:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc000, rc 0) 00000100:00100000:0.0:1788572572.795521:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:2348:x1875453360647936:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 7151us (11148us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572572.795546:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 406 00010000:00010000:3.0:1788572572.795787:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572572.802153:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:37246:x1875453360648320:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 8177us (8362us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572572.802182:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4888 00000100:00100000:2.0:1788572572.811335:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572572.811450:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360648448 00000100:00100000:3.0:1788572572.811464:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360648448 00000100:00100000:3.0:1788572572.811539:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4889 00000100:00100000:3.0:1788572572.811552:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+57:37246:x1875453360648448:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:3.0:1788572572.811590:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572572.811621:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165bf4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1647 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811727:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811746:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811760:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811780:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811788:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811795:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811911:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165c09 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811918:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165c09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.811923:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165c09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572572.811963:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:3.0:1788572572.811968:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572572.811989:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165c09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812035:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812041:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812046:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812050:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572572.812053:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812058:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812069:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627ff5ab200) 00010000:00010000:3.0:1788572572.812071:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572572.812074:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812079:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb165bfb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812089:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812092:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812097:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812103:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572572.812106:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812112:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812120:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54976cc (ffff9626c4b10500) 00010000:00010000:3.0:1788572572.812122:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572572.812125:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812132:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb165c02 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812161:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165bf4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1647 expref: 59 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812172:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165c09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1647 expref: 58 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.812177:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:3.0:1788572572.812178:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572572.812180:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4000, rc 0) 00000100:00100000:3.0:1788572572.812251:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:37246:x1875453360648448:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 703us (935us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572572.812257:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4889 00000100:00100000:2.0:1788572572.824744:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572572.824880:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360648576 00000100:00100000:1.0:1788572572.824895:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360648576 00000100:00100000:1.0:1788572572.825028:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4890 00000100:00100000:1.0:1788572572.825043:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:37248:x1875453360648576:12345-192.168.202.40@tcp:101:rm.0 00010000:00010000:0.0:1788572572.825544:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572572.825576:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165c10 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f164e expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825717:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825734:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825749:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825772:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825780:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825787:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825882:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825890:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825897:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825903:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.825907:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825915:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825929:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:0.0:1788572572.825932:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.825937:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825944:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c17 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825960:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825966:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825972:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825978:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.825981:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825987:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.825994:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627c5160900) 00010000:00010000:0.0:1788572572.825996:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.825999:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.826060:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c1e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.826104:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165c10 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f164e expref: 59 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.826115:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:0.0:1788572572.826118:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.826122:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb165c10 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f164e expref: 59 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.826132:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646c00, rc 0) 00000100:00100000:0.0:1788572572.826216:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:37248:x1875453360648576:12345-192.168.202.40@tcp:101:rm.0 Request processed in 1176us (1486us total) trans 0 rc 301/301 00000100:00100000:0.0:1788572572.826226:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4890 00000100:00100000:0.0:1788572572.842752:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572572.842938:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360648832 00000100:00100000:2.0:1788572572.842952:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360648832 00000100:00100000:2.0:1788572572.843155:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4891 00000100:00100000:2.0:1788572572.843171:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:37249:x1875453360648832:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:2.0:1788572572.843200:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572572.843205:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788572572.843238:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb165c09 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1647 expref: 58 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.843262:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:2.0:1788572572.843276:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572572.843295:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165c25 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1655 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.844409:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.844435:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.844451:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.844470:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.844477:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.844484:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572572.847233:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5c:0x0], index 0: 0 00010000:00010000:3.0:1788572572.847575:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165c3a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847599:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165c3a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847613:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165c3a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847742:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847751:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847759:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847765:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572572.847771:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847779:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847796:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627d20cb300) 00010000:00010000:3.0:1788572572.847800:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572572.847805:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.847811:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb165c2c lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572572.847832:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000305a372e reply state 0000000044de4e79 transno 4294970996 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572572.847841:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572572.847988:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb165c25 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1655 expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.848243:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165c3a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1655 expref: 59 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572572.848253:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:3.0:1788572572.848256:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572572.848259:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4400, rc 0) 00000100:00100000:3.0:1788572572.848975:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37249:x1875453360648832:12345-192.168.202.40@tcp:101:dd.0 Request processed in 5802us (6232us total) trans 4294970996 rc 0/0 00000100:00100000:3.0:1788572572.848987:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4891 00010000:00010000:0.0:1788572572.859278:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.859302:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.859311:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.859318:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572572.859324:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.859333:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.859354:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572572.859358:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572572.859364:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572572.859371:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165c33 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572572.881926:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572572.882103:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360648960 00000100:00100000:2.0:1788572572.882188:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360648960 00000100:00100000:2.0:1788572572.882277:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1420 00000100:00100000:2.0:1788572572.882290:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:37249:x1875453360648960:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:2.0:1788572572.882314:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572572.882345:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f165c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572572.882381:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1788572572.882411:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=920s v=(0 0) 00010000:00010000:2.0:1788572572.882416:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572572.882429:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.882530:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 10 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572572.882555:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8a00, rc 0) 00000100:00100000:2.0:1788572572.882722:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:37249:x1875453360648960:12345-192.168.202.40@tcp:101:dd.0 Request processed in 432us (812us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572572.882730:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1420 00000100:00100000:2.0:1788572572.899728:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572572.899836:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649088 00000100:00100000:1.0:1788572572.899852:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649088 00002000:00100000:1.0:1788572572.899921:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5c:0x0] (0->1048575) req@ffff9627e786f480 x1875453360649088/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:233/0 lens 488/0 e 0 to 0 dl 1788572583 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572572.899934:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786f480 with x1875453360649088 ext(0->1048575) 00002000:00010000:1.0:1788572572.899942:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786f480 00000100:00100000:1.0:1788572572.900852:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 362 00000100:00100000:1.0:1788572572.900873:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360649088:12345-192.168.202.40@tcp:4:dd.0 00000100:00100000:0.0:1788572572.930459:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360649088:12345-192.168.202.40@tcp:4:dd.0 Request processed in 29578us (30732us total) trans 4294967673 rc 0/0 00000100:00100000:0.0:1788572572.930474:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 362 00002000:00100000:0.0:1788572572.930487:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5c:0x0] (0->1048575) req@ffff9627e786f480 x1875453360649088/t4294967673(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:233/0 lens 488/448 e 0 to 0 dl 1788572583 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572572.930497:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786f480 with x1875453360649088 ext(0->1048575) 00002000:00010000:0.0:1788572572.930505:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786f480 00000100:00100000:1.0:1788572572.935207:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572572.935315:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649216 00000100:00100000:3.0:1788572572.935332:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649216 00000100:00100000:3.0:1788572572.935431:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4892 00000100:00100000:3.0:1788572572.935444:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:37249:x1875453360649216:12345-192.168.202.40@tcp:44:dd.0 00000100:00100000:2.0:1788572572.987187:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572572.987216:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382402304:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788572572.987310:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572572.987401:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382402304 00000100:00100000:2.0:1788572572.987411:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382402304 00000100:00100000:2.0:1788572572.987487:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:2.0:1788572572.987498:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382402304:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788572572.987527:0:6684:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 195493888 pending 1703936 free 3858759680 avail 3856662528 00000100:00100000:2.0:1788572572.987578:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382402304:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572572.987587:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00080000:2.0:1788572572.987633:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572572.987666:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967672 gen 2 00000004:00080000:2.0:1788572572.987673:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627d0fb0a80, transno 4294967672 00000100:00100000:2.0:1788572572.987693:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382402304:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788572572.992816:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:37249:x1875453360649216:12345-192.168.202.40@tcp:44:dd.0 Request processed in 57358us (57604us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572572.992833:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4892 00000020:00080000:0.0:1788572572.994808:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 146 batchid updated 00000001:00080000:0.0:1788572572.994849:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970996 is committed 00000001:00080000:0.0:1788572572.994857:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970995 is committed 00000020:00080000:2.0:1788572572.994931:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 146 00000020:00080000:2.0:1788572572.994946:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 146 00000100:00100000:2.0:1788572572.995947:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572572.996072:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649344 00000100:00100000:0.0:1788572572.996087:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649344 00000100:00100000:0.0:1788572572.996164:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1421 00000100:00100000:0.0:1788572572.996176:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360649344:12345-192.168.202.40@tcp:16:dd.0 00000100:00100000:3.0:1788572573.032664:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360649344:12345-192.168.202.40@tcp:16:dd.0 Request processed in 36473us (36711us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.032683:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1421 00000001:00080000:1.0:1788572573.032794:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967673 is committed 00000100:00100000:3.0:1788572573.035554:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.035684:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649600 00000100:00100000:0.0:1788572573.035698:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649600 00000100:00100000:0.0:1788572573.035799:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1061 00000100:00100000:0.0:1788572573.035811:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec038450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+58:37249:x1875453360649600:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:0.0:1788572573.036808:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec038450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:37249:x1875453360649600:12345-192.168.202.40@tcp:35:dd.0 Request processed in 992us (1261us total) trans 4294970997 rc 0/0 00000100:00100000:0.0:1788572573.036819:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1061 00000100:00100000:0.0:1788572573.056567:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.056711:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649728 00000100:00100000:2.0:1788572573.056726:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649728 00000100:00100000:2.0:1788572573.056810:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4893 00000100:00100000:2.0:1788572573.056822:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+59:37250:x1875453360649728:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572573.056862:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572573.056887:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165c4f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1663 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.056974:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.056999:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057036:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057050:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057056:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057059:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057605:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165c64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057624:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165c64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057635:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165c64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572573.057671:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572573.057676:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572573.057702:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165c64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057716:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057724:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057732:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057739:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.057744:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057753:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057767:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627ff5ab200) 00010000:00010000:2.0:1788572573.057770:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.057777:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057784:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165c56 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057801:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057807:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057814:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057819:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.057822:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057829:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.057836:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54976cc (ffff9627ffc19900) 00010000:00010000:2.0:1788572573.057838:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.057841:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.058200:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c5d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.058237:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165c4f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1663 expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.058248:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9a00/0x85ace67feb165c64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1663 expref: 60 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.058258:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572573.058260:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1788572573.058262:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9a00, rc 0) 00000100:00100000:2.0:1788572573.058335:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37250:x1875453360649728:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1516us (1781us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.058341:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4893 00000100:00100000:3.0:1788572573.085737:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.085859:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649856 00000100:00100000:0.0:1788572573.085874:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649856 00000100:00100000:0.0:1788572573.085977:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4894 00000100:00100000:0.0:1788572573.086048:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+239:37253:x1875453360649856:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572573.086248:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.086284:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c6b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f166a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.086376:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c72 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.086394:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.086408:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572573.086455:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788572573.086459:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572573.086482:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.086511:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165c6b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f166a expref: 241 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.086532:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646e00/0x85ace67feb165c72 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f166a expref: 240 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.086542:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627ff5ab200) 00010000:00010000:0.0:1788572573.086545:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572573.086549:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646e00, rc 0) 00000100:00100000:0.0:1788572573.086638:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:37253:x1875453360649856:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 651us (921us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.086645:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4894 00000100:00080000:2.0:1788572573.087602:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788572573.093188:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.093477:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360649984 00000100:00100000:1.0:1788572573.093491:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360649984 00000100:00100000:1.0:1788572573.093572:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4895 00000100:00100000:1.0:1788572573.093585:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+240:37253:x1875453360649984:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:1.0:1788572573.093628:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572573.093660:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165c79 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1671 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.093761:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.093779:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.093793:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.093811:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.093817:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.093823:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094650:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165c8e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094669:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165c8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094678:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165c8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572573.094721:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:1.0:1788572573.094725:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572573.094752:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165c8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094768:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094775:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094784:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094791:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572573.094796:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094806:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094822:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9627ff5ab200) 00010000:00010000:1.0:1788572573.094827:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572573.094834:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.094841:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb165c80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096604:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096627:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096635:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096640:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572573.096644:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096652:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096667:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54976cc (ffff9627ff5ab700) 00010000:00010000:1.0:1788572573.096670:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572573.096675:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096683:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165c87 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54976cc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096744:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165c79 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1671 expref: 242 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096765:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bdc00/0x85ace67feb165c8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1671 expref: 241 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.096773:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572573.096775:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1788572573.096778:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bdc00, rc 0) 00000100:00100000:2.0:1788572573.097322:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:37253:x1875453360649984:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 3735us (4146us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.097335:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4895 00000100:00100000:1.0:1788572573.100947:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.101670:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360650240 00000100:00100000:2.0:1788572573.101685:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360650240 00000100:00100000:2.0:1788572573.101771:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4896 00000100:00100000:2.0:1788572573.101783:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+241:37253:x1875453360650240:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572573.101813:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572573.101842:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165c95 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1678 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.101960:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.101979:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.101993:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102064:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102071:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102077:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102812:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 3/1,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102825:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102833:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572573.102845:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xab:0x0]: returning size 1048576 00010000:00010000:2.0:1788572573.102981:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102990:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.102998:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103038:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.103044:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103059:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103075:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572573.103079:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.103095:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103916:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb165c9c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103936:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103942:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103949:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103954:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.103957:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103964:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103976:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627ffc19900) 00010000:00010000:2.0:1788572573.103979:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.103983:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.103990:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb165ca3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.104293:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165c95 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 244 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.104312:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 243 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.104321:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:2.0:1788572573.104323:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572573.104327:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9c00, rc 0) 00000100:00100000:2.0:1788572573.104545:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37253:x1875453360650240:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 2762us (3614us total) trans 4294970998 rc 0/0 00000100:00100000:2.0:1788572573.104556:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4896 00000100:00100000:3.0:1788572573.108749:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.108838:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360650368 00000100:00100000:0.0:1788572573.108852:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360650368 00000100:00100000:0.0:1788572573.108978:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1422 00000100:00100000:0.0:1788572573.108991:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2350:x1875453360650368:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572573.109038:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.109061:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f85d513/0x85ace67feb165cb8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f167f expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.109609:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f5969500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382402432:192.168.202.40@tcp:106: 00000100:00100000:0.0:1788572573.109780:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f64d80 going to sleep for 11 seconds 00000100:00080000:2.0:1788572573.109913:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1788572573.112302:0:6679:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x5c:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:0.0:1788572573.112314:0:6679:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x5c:0x0] updating lvb mtime: 0 -> 1788572572 00002000:00010000:0.0:1788572573.112318:0:6679:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x5c:0x0] updating lvb atime: 0 -> 1788572572 00002000:00010000:0.0:1788572573.112320:0:6679:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x5c:0x0] updating lvb ctime: 0 -> 1788572572 00002000:00010000:0.0:1788572573.112322:0:6679:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x5c:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:0.0:1788572573.112352:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f5969500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382402432:192.168.202.40@tcp:106: 00010000:00010000:0.0:1788572573.112415:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f85d513/0x85ace67feb165cb8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f167f expref: 72 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.112447:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f85d513/0x85ace67feb165cb8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f167f expref: 72 pid: 6679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.112463:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646400, rc 0) 00000100:00100000:0.0:1788572573.112768:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:2350:x1875453360650368:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 3777us (4034us total) trans 0 rc 301/301 00000100:00100000:0.0:1788572573.112778:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1422 00000100:00100000:0.0:1788572573.116329:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.116786:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360650496 00000100:00100000:3.0:1788572573.116799:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360650496 00000100:00100000:3.0:1788572573.116888:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1423 00000100:00100000:3.0:1788572573.116901:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+71:37253:x1875453360650496:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572573.116926:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572573.116953:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1686 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.117000:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.117032:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x5c:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788572573.117037:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572573.117044:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 4/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.117315:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 9 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.117333:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 9 pid: 6682 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.117353:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382402560:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572573.117470:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572573.121909:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.121994:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360650624 00000100:00100000:0.0:1788572573.122042:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360650624 00010000:00010000:0.0:1788572573.122132:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 10 pid: 6682 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.122183:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1143 00000100:00100000:0.0:1788572573.122196:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360650624:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572573.122219:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572573.122251:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 10 pid: 6682 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.122267:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 10 pid: 6682 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.122303:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572573.122308:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572573.122318:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.122329:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.122397:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.122416:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382402688:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572573.122533:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f64d80 going to sleep for 11 seconds 00000100:00080000:3.0:1788572573.123273:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572573.125510:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382402688:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572573.125549:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572573.125640:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360650624:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3444us (3744us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.125654:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1143 00010000:00010000:2.0:1788572573.127051:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165c48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f165c expref: 9 pid: 6682 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.127079:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382402560:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572573.127485:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.127502:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4200, rc 0) 00000100:00100000:2.0:1788572573.127586:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:37253:x1875453360650496:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 10688us (11269us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.127597:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1423 00000100:00100000:3.0:1788572573.130830:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.130934:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360650752 00000100:00100000:2.0:1788572573.130947:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360650752 00002000:00100000:2.0:1788572573.131039:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5c:0x0] (0->32767) req@ffff9627d131fb80 x1875453360650752/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.131053:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131fb80 with x1875453360650752 ext(0->32767) 00002000:00010000:2.0:1788572573.131060:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131fb80 00000100:00100000:2.0:1788572573.131237:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 363 00000100:00100000:2.0:1788572573.131251:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2348:x1875453360650752:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572573.135832:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2348:x1875453360650752:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 4568us (5002us total) trans 0 rc 32768/32768 00000100:00100000:2.0:1788572573.135850:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 363 00002000:00100000:2.0:1788572573.135867:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5c:0x0] (0->32767) req@ffff9627d131fb80 x1875453360650752/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/440 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.135879:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131fb80 with x1875453360650752 ext(0->32767) 00002000:00010000:2.0:1788572573.135886:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131fb80 00000100:00100000:3.0:1788572573.142812:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.142932:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360650880 00000100:00100000:2.0:1788572573.142946:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360650880 00002000:00100000:2.0:1788572573.143057:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5c:0x0] (32768->1048575) req@ffff9627d131f480 x1875453360650880/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.143071:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131f480 with x1875453360650880 ext(32768->1048575) 00002000:00010000:2.0:1788572573.143079:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131f480 00000100:00100000:2.0:1788572573.143124:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 364 00000100:00100000:2.0:1788572573.143136:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2349:x1875453360650880:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572573.158530:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2349:x1875453360650880:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 15378us (15717us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572573.158548:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 364 00002000:00100000:2.0:1788572573.158564:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5c:0x0] (32768->1048575) req@ffff9627d131f480 x1875453360650880/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/440 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.158576:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131f480 with x1875453360650880 ext(32768->1048575) 00002000:00010000:2.0:1788572573.158585:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131f480 00000100:00100000:0.0:1788572573.167499:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.167623:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360651136 00000100:00100000:2.0:1788572573.167637:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360651136 00000100:00100000:2.0:1788572573.167713:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1062 00000100:00100000:2.0:1788572573.167725:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37253:x1875453360651136:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:2.0:1788572573.168392:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37253:x1875453360651136:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 666us (904us total) trans 4294970999 rc 0/0 00000100:00100000:2.0:1788572573.168404:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1062 00000100:00100000:3.0:1788572573.187164:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.187565:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360651264 00000100:00100000:0.0:1788572573.187580:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360651264 00000100:00100000:0.0:1788572573.187690:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4897 00000100:00100000:0.0:1788572573.187705:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37255:x1875453360651264:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572573.187739:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.187767:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165cc6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f168d expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.187942:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.187961:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.187975:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572573.190032:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572573.190047:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572573.190287:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.190343:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb165cc6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 246 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.190519:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 245 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.190530:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572573.190533:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572573.190539:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646000, rc 0) 00000100:00100000:0.0:1788572573.190902:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37255:x1875453360651264:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 3198us (3752us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.190914:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4897 00000100:00100000:0.0:1788572573.199734:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.199898:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360651520 00000100:00100000:2.0:1788572573.199912:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360651520 00000100:00100000:2.0:1788572573.200265:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4898 00000100:00100000:2.0:1788572573.200280:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37255:x1875453360651520:12345-192.168.202.40@tcp:58:lfs.0 00000040:00080000:2.0:1788572573.201487:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].169, 136 off30968 00000100:00100000:2.0:1788572573.201712:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37255:x1875453360651520:12345-192.168.202.40@tcp:58:lfs.0 Request processed in 1430us (1987us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.201724:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4898 00000100:00100000:3.0:1788572573.236236:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.236336:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360651648 00000100:00100000:1.0:1788572573.236350:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360651648 00000100:00100000:1.0:1788572573.236429:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4899 00000100:00100000:1.0:1788572573.236442:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec191050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37259:x1875453360651648:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:1.0:1788572573.236472:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572573.236502:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165cd4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1694 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.236597:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165cdb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.236625:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165cdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.236640:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165cdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572573.236686:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:1.0:1788572573.236690:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572573.236718:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165cdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.236751:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165cd4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1694 expref: 62 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.236773:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165cdb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1694 expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.236782:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572573.236786:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572573.236791:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bde00, rc 0) 00000100:00100000:1.0:1788572573.237619:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec191050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37259:x1875453360651648:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 1163us (1616us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.237631:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4899 00000100:00100000:3.0:1788572573.253826:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.254062:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360651776 00000100:00100000:0.0:1788572573.254086:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360651776 00000100:00100000:0.0:1788572573.254393:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4900 00000100:00100000:0.0:1788572573.254408:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37260:x1875453360651776:12345-192.168.202.40@tcp:101:rm.0 00010000:00010000:0.0:1788572573.254440:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.254470:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ce2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f169b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.254559:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.254579:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.254594:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.254623:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.254631:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.254638:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255045:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255056:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255064:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255071:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.255260:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255272:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255290:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572573.255293:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.255300:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255308:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb165ce9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255324:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255331:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255338:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255344:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.255347:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255354:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255362:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9627c5160900) 00010000:00010000:0.0:1788572573.255365:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.255368:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255375:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb165cf0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255422:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ce2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f169b expref: 62 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255441:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572573.255444:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.255449:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb165ce2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f169b expref: 62 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.255459:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647200, rc 0) 00000100:00100000:0.0:1788572573.255910:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37260:x1875453360651776:12345-192.168.202.40@tcp:101:rm.0 Request processed in 1500us (2092us total) trans 0 rc 301/301 00000100:00100000:0.0:1788572573.255920:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4900 00000100:00100000:0.0:1788572573.274589:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.274716:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652032 00000100:00100000:3.0:1788572573.274729:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652032 00000100:00100000:3.0:1788572573.274866:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4901 00000100:00100000:3.0:1788572573.274879:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37261:x1875453360652032:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:3.0:1788572573.274909:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572573.274913:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788572573.274949:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bde00/0x85ace67feb165cdb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1694 expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.274975:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572573.274982:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572573.275001:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165cf7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16a2 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.275768:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.275791:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 244 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.275800:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.275843:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 244 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.275862:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 244 pid: 23058 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.275878:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382402816:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572573.276052:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0750c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572573.280765:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382402816:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572573.280815:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.282358:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.282745:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652160 00000100:00100000:1.0:1788572573.282756:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652160 00010000:00010000:1.0:1788572573.282809:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 245 pid: 23058 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572573.282981:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1144 00000100:00100000:1.0:1788572573.282993:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:2350:x1875453360652160:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572573.283041:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572573.283051:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 245 pid: 23058 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.283072:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 245 pid: 23058 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.283114:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb165ccd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f168d expref: 245 pid: 23058 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.283134:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572573.283148:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.283161:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.283170:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.283205:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572573.283284:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:2350:x1875453360652160:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 292us (936us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.283292:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1144 00010000:00010000:3.0:1788572573.283660:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.283671:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.283702:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.283714:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.283723:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572573.286789:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x5c:0x0], index 1: 0 00010000:00010000:3.0:1788572573.288059:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb165d0c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288476:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb165d0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288494:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb165d0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288552:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288560:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288567:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288573:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572573.288579:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288588:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288602:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572573.288616:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572573.288622:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288629:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165cfe lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572573.288645:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000046542769 reply state 0000000070cded2e transno 4294971000 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.288655:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572573.288703:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165cf7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a2 expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288726:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb165d0c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a2 expref: 62 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.288734:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:3.0:1788572573.288737:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572573.288740:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae600, rc 0) 00000100:00100000:3.0:1788572573.288841:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37261:x1875453360652032:12345-192.168.202.40@tcp:101:dd.0 Request processed in 13963us (14265us total) trans 4294971000 rc 0/0 00000100:00100000:3.0:1788572573.288864:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4901 00010000:00010000:2.0:1788572573.294988:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.295050:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.295060:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.295067:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.295074:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.295084:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.295107:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9626c4b10b00) 00010000:00010000:2.0:1788572573.295111:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.295118:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.295125:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165d05 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.310341:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.310583:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652288 00000100:00100000:0.0:1788572573.310597:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652288 00000100:00100000:0.0:1788572573.310702:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1424 00000100:00100000:0.0:1788572573.310717:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:37261:x1875453360652288:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572573.310741:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.310770:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16a9 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572573.310805:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572573.310837:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=921s v=(0 0) 00010000:00010000:0.0:1788572573.310843:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572573.310857:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 10 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.310887:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 10 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.310903:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646a00, rc 0) 00000100:00100000:0.0:1788572573.310990:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:37261:x1875453360652288:12345-192.168.202.40@tcp:101:dd.0 Request processed in 278us (666us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.310999:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1424 00000100:00080000:3.0:1788572573.312241:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788572573.331975:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.332218:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652416 00000100:00100000:1.0:1788572573.332232:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652416 00002000:00100000:1.0:1788572573.332292:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5c:0x0] (0->1048575) req@ffff9626c1d88e00 x1875453360652416/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572573.332306:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1d88e00 with x1875453360652416 ext(0->1048575) 00002000:00010000:1.0:1788572573.332312:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1d88e00 00000100:00100000:1.0:1788572573.332350:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 365 00000100:00100000:1.0:1788572573.332369:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360652416:12345-192.168.202.40@tcp:4:dd.0 00000100:00100000:3.0:1788572573.415249:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360652416:12345-192.168.202.40@tcp:4:dd.0 Request processed in 82861us (83268us total) trans 4294967674 rc 0/0 00000100:00100000:3.0:1788572573.415273:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 365 00002000:00100000:3.0:1788572573.415291:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5c:0x0] (0->1048575) req@ffff9626c1d88e00 x1875453360652416/t4294967674(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/448 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572573.415303:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1d88e00 with x1875453360652416 ext(0->1048575) 00002000:00010000:3.0:1788572573.415311:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c1d88e00 00000100:00100000:0.0:1788572573.418857:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.418968:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652544 00000100:00100000:2.0:1788572573.418982:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652544 00000100:00100000:2.0:1788572573.419282:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4902 00000100:00100000:2.0:1788572573.419295:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37261:x1875453360652544:12345-192.168.202.40@tcp:44:dd.0 00000001:00080000:0.0:1788572573.456217:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970997 is committed 00000001:00080000:0.0:1788572573.456246:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294970999 is committed 00000001:00080000:0.0:1788572573.456256:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971000 is committed 00000100:00100000:2.0:1788572573.456549:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37261:x1875453360652544:12345-192.168.202.40@tcp:44:dd.0 Request processed in 37248us (37698us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.456563:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4902 00000100:00100000:0.0:1788572573.459038:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.459173:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652672 00000100:00100000:1.0:1788572573.459187:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652672 00000100:00100000:1.0:1788572573.459271:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1425 00000100:00100000:1.0:1788572573.459283:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360652672:12345-192.168.202.40@tcp:16:dd.0 00000100:00100000:2.0:1788572573.498609:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3304:1875453382403072:0@lo:400:kworker.0 00000100:00100000:2.0:1788572573.498710:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572573.498780:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1788572573.498787:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3304:1875453382403200:0@lo:400:kworker.0 00000100:00100000:2.0:1788572573.498815:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572573.498998:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3303:1875453382402944:0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499086:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572573.499110:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382403456:0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499139:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572573.499161:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1788572573.499167:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3303:1875453382403328:0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499189:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572573.499224:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382403072 00000100:00100000:3.0:1788572573.499234:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382403072 00000100:00100000:3.0:1788572573.499287:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382403200 00000100:00100000:3.0:1788572573.499290:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382403200 00000100:00100000:3.0:1788572573.499324:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382403328 00000100:00100000:3.0:1788572573.499327:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382403328 00000100:00100000:2.0:1788572573.499511:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382402944 00000100:00100000:2.0:1788572573.499516:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382402944 00000100:00100000:2.0:1788572573.499552:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:2.0:1788572573.499563:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3303:x1875453382402944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572573.499636:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3303:x1875453382402944:12345-0@lo:400:kworker.0 Request processed in 75us (550us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.499644:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:3.0:1788572573.499731:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:3.0:1788572573.499738:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3304:x1875453382403072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499783:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3304:x1875453382403072:12345-0@lo:400:kworker.0 Request processed in 46us (1074us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.499790:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:3.0:1788572573.499804:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:3.0:1788572573.499811:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1875453382403200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499837:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1875453382403200:12345-0@lo:400:kworker.0 Request processed in 28us (1022us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.499843:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:3.0:1788572573.499854:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:3.0:1788572573.499860:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1875453382403328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499888:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1875453382403328:12345-0@lo:400:kworker.0 Request processed in 28us (698us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.499893:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00080000:3.0:1788572573.499918:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788572573.499963:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3304:1875453382403200:0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.499978:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3304:1875453382403072:0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.500044:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3303:1875453382403328:0@lo:400:kworker.0 00000100:00100000:3.0:1788572573.500055:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3303:1875453382402944:0@lo:400:kworker.0 00000100:00100000:0.0:1788572573.501132:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382403456 00000100:00100000:0.0:1788572573.501143:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382403456 00000100:00100000:0.0:1788572573.501214:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:0.0:1788572573.501226:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382403456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788572573.501302:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382403456:12345-0@lo:400:kworker.0 Request processed in 80us (2161us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.501309:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:0.0:1788572573.501373:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967673 gen 2 00000004:00080000:0.0:1788572573.501379:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627c9a5aa00, transno 4294967671 00000004:00080000:0.0:1788572573.501391:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967673 committed transno 4294967673 00000100:00100000:0.0:1788572573.501435:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e786ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382403456:0@lo:400:kworker.0 00000100:00080000:3.0:1788572573.503591:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572573.522506:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360652672:12345-192.168.202.40@tcp:16:dd.0 Request processed in 63210us (63486us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.522524:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1425 00000100:00100000:0.0:1788572573.525177:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000001:00080000:3.0:1788572573.525571:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967674 is committed 00000100:00100000:1.0:1788572573.525713:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360652928 00000100:00100000:1.0:1788572573.525727:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360652928 00000100:00100000:1.0:1788572573.525799:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1063 00000100:00100000:1.0:1788572573.525810:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37261:x1875453360652928:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:1.0:1788572573.526491:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37261:x1875453360652928:12345-192.168.202.40@tcp:35:dd.0 Request processed in 678us (1324us total) trans 4294971001 rc 0/0 00000100:00100000:1.0:1788572573.526500:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1063 00000100:00100000:0.0:1788572573.569143:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.569339:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653184 00000100:00100000:3.0:1788572573.569365:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653184 00000100:00100000:3.0:1788572573.569442:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4903 00000100:00100000:3.0:1788572573.569454:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:37265:x1875453360653184:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572573.569483:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572573.569513:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165d21 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16b0 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.569631:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.569650:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.569663:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.569680:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.569687:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.569694:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570520:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570540:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570548:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572573.570561:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xac:0x0]: returning size 1048576 00010000:00010000:3.0:1788572573.570704:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570713:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570721:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570728:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572573.570734:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570744:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570759:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572573.570763:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572573.570770:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570778:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570799:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570805:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570811:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570817:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aec00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572573.570820:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570825:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570832:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9626c4b10a00) 00010000:00010000:3.0:1788572573.570835:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572573.570838:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.570844:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.571001:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165d21 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 245 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.571963:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 244 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.571979:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:3.0:1788572573.571981:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572573.571985:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af200, rc 0) 00000100:00100000:3.0:1788572573.572586:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37265:x1875453360653184:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 3129us (3455us total) trans 4294971002 rc 0/0 00000100:00100000:3.0:1788572573.572607:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4903 00000100:00100000:0.0:1788572573.582997:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.583174:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653312 00000100:00100000:2.0:1788572573.583187:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653312 00000100:00100000:2.0:1788572573.583267:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1426 00000100:00100000:2.0:1788572573.583280:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360653312:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572573.583303:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572573.583329:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001cefab49/0x85ace67feb165d44 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16b7 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.583424:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382403584:192.168.202.40@tcp:106: 00000100:00100000:2.0:1788572573.583536:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572573.583820:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.584242:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653440 00000100:00100000:2.0:1788572573.584249:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653440 00000100:00100000:2.0:1788572573.584304:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 408 00000100:00100000:2.0:1788572573.584314:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:2347:x1875453360653440:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572573.584386:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572573.584406:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165d4b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16be expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584474:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584492:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584506:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584521:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584527:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584533:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584810:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584820:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584827:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584834:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.584841:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584850:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584865:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572573.584868:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.584875:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584881:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb165d52 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584895:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584901:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584907:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584912:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572573.584915:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584921:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584927:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627e3306800) 00010000:00010000:2.0:1788572573.584930:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572573.584933:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584939:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb165d59 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584973:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165d4b lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16be expref: 245 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.584985:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572573.584988:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572573.584991:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb165d4b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16be expref: 245 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.585001:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8a00, rc -2) 00000100:00100000:2.0:1788572573.585139:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:2347:x1875453360653440:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 827us (1319us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572573.585148:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 408 00000100:00080000:1.0:1788572573.586679:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1788572573.589956:0:10416:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x5c:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:2.0:1788572573.589970:0:10416:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x5c:0x0] updating lvb mtime: 0 -> 1788572572 00002000:00010000:2.0:1788572573.589973:0:10416:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x5c:0x0] updating lvb atime: 0 -> 1788572572 00002000:00010000:2.0:1788572573.589975:0:10416:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x5c:0x0] updating lvb ctime: 0 -> 1788572572 00002000:00010000:2.0:1788572573.589977:0:10416:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x5c:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:2.0:1788572573.590044:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382403584:192.168.202.40@tcp:106: 00010000:00010000:2.0:1788572573.590107:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001cefab49/0x85ace67feb165d44 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b7 expref: 84 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.590139:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001cefab49/0x85ace67feb165d44 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b7 expref: 84 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.590157:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8c00, rc 0) 00000100:00100000:2.0:1788572573.590236:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2350:x1875453360653312:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 6958us (7253us total) trans 0 rc 301/301 00000100:00100000:2.0:1788572573.590247:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1426 00000100:00100000:3.0:1788572573.596513:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.596762:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653568 00000100:00100000:0.0:1788572573.596775:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653568 00000100:00100000:0.0:1788572573.596888:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1427 00000100:00100000:0.0:1788572573.596904:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:37265:x1875453360653568:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572573.596932:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.596960:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16f6 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.597052:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 9 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.597071:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572573.597078:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572573.597085:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.597130:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 9 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.597147:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 9 pid: 10416 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.597164:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382403712:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572573.597268:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572573.601220:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.601310:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653696 00000100:00100000:2.0:1788572573.601320:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653696 00010000:00010000:2.0:1788572573.601382:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 10 pid: 10416 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.601427:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1145 00000100:00100000:2.0:1788572573.601438:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360653696:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572573.601457:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572573.601484:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 10 pid: 10416 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.601500:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 10 pid: 10416 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.601531:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.601536:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.601547:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.601559:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.601619:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.601636:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382403840:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572573.601733:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56840 going to sleep for 11 seconds 00000100:00100000:2.0:1788572573.604077:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382403840:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572573.604114:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572573.604179:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360653696:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 2742us (2971us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.604192:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1145 00010000:00010000:3.0:1788572573.605359:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb058e53/0x85ace67feb165d1a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a9 expref: 9 pid: 10416 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.605384:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382403712:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572573.605441:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.605457:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647a00, rc 0) 00000100:00100000:3.0:1788572573.605513:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:37265:x1875453360653568:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 8615us (9020us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.605523:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1427 00000100:00100000:1.0:1788572573.608819:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.609114:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653824 00000100:00100000:2.0:1788572573.609127:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653824 00002000:00100000:2.0:1788572573.609196:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5c:0x0] (0->32767) req@ffff9627c9b52d80 x1875453360653824/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.609209:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9b52d80 with x1875453360653824 ext(0->32767) 00002000:00010000:2.0:1788572573.609215:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9b52d80 00000100:00100000:2.0:1788572573.609250:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 366 00000100:00100000:2.0:1788572573.609261:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2347:x1875453360653824:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572573.614889:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2347:x1875453360653824:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 5612us (6067us total) trans 0 rc 32768/32768 00000100:00100000:2.0:1788572573.614910:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 366 00002000:00100000:2.0:1788572573.614928:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5c:0x0] (0->32767) req@ffff9627c9b52d80 x1875453360653824/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/440 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.614948:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c9b52d80 with x1875453360653824 ext(0->32767) 00002000:00010000:2.0:1788572573.614956:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c9b52d80 00000100:00100000:0.0:1788572573.623233:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.623421:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360653952 00000100:00100000:2.0:1788572573.623436:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360653952 00002000:00100000:2.0:1788572573.623512:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5c:0x0] (32768->1048575) req@ffff9627ff5ef480 x1875453360653952/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.623526:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff5ef480 with x1875453360653952 ext(32768->1048575) 00002000:00010000:2.0:1788572573.623534:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ff5ef480 00000100:00100000:2.0:1788572573.623728:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 367 00000100:00100000:2.0:1788572573.623742:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360653952:12345-192.168.202.40@tcp:3:md5sum.0 00000040:00080000:1.0:1788572573.626576:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572573.626590:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572573.626662:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 170 count 4 00000040:00080000:1.0:1788572573.626672:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1788572573.627201:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382403968:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572573.627325:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572573.631790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.631893:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382403968:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572573.632245:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360654080 00000100:00100000:3.0:1788572573.632254:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360654080 00000100:00100000:3.0:1788572573.632317:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4904 00000100:00100000:3.0:1788572573.632327:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37269:x1875453360654080:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572573.632376:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xab:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572573.632485:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37269:x1875453360654080:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 159us (709us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.632493:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4904 00000100:00100000:3.0:1788572573.653033:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360653952:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 29248us (29768us total) trans 0 rc 1015808/1015808 00000100:00100000:3.0:1788572573.653052:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 367 00002000:00100000:3.0:1788572573.653078:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5c:0x0] (32768->1048575) req@ffff9627ff5ef480 x1875453360653952/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:234/0 lens 488/440 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572573.653090:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff5ef480 with x1875453360653952 ext(32768->1048575) 00002000:00010000:3.0:1788572573.653099:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ff5ef480 00000100:00100000:2.0:1788572573.654606:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.654751:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360654336 00000100:00100000:0.0:1788572573.654764:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360654336 00000100:00100000:0.0:1788572573.654863:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4905 00000100:00100000:0.0:1788572573.654877:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37269:x1875453360654336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572573.654907:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.654936:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165d67 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f16fd expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.655128:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165d6e lrc: 3/1,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.655146:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165d6e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.655162:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165d6e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572573.655176:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xab:0x0]: returning size 1048576 00010000:00010000:0.0:1788572573.655354:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165d67 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16fd expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.655375:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165d6e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16fd expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.655385:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:0.0:1788572573.655388:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572573.655393:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f200, rc 0) 00000100:00100000:0.0:1788572573.655457:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37269:x1875453360654336:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 584us (874us total) trans 4294971003 rc 0/0 00000100:00100000:0.0:1788572573.655464:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4905 00000100:00100000:0.0:1788572573.663163:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.663325:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360654464 00000100:00100000:2.0:1788572573.663339:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360654464 00000100:00100000:2.0:1788572573.663405:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1428 00000100:00100000:2.0:1788572573.663416:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360654464:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572573.663476:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663496:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.663500:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.663508:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663524:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663564:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663572:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663580:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663587:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572573.663603:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663613:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663630:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.663639:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008805b04f/0x85ace67feb165d7c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.663740:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360654464:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 325us (592us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.663748:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1428 00000100:00100000:1.0:1788572573.666722:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.666832:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360654592 00000100:00100000:0.0:1788572573.666845:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360654592 00000100:00100000:0.0:1788572573.666959:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 375 00000100:00100000:0.0:1788572573.666973:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:2350:x1875453360654592:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572573.666995:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572573.667083:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165d6e lrc: 0/0,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16fd expref: 63 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.667109:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:0.0:1788572573.667116:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572573.667183:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360654592:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 213us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.667191:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 375 00000100:00100000:0.0:1788572573.672647:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.672795:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360654848 00000100:00100000:2.0:1788572573.672810:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360654848 00000100:00100000:2.0:1788572573.672904:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4906 00000100:00100000:2.0:1788572573.672917:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37269:x1875453360654848:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572573.673174:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37269:x1875453360654848:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 259us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.673183:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4906 00000100:00100000:0.0:1788572573.677150:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.679609:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.679696:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360655232 00000100:00100000:1.0:1788572573.679709:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360655232 00000100:00100000:1.0:1788572573.679777:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1429 00000100:00100000:1.0:1788572573.679787:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360655232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572573.679811:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572573.679834:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1704 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.679876:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572573.679902:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572573.679915:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 10 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.679937:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 10 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.679953:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bce00, rc 0) 00000100:00100000:1.0:1788572573.680037:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360655232:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 250us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.680045:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1429 00000100:00100000:3.0:1788572573.680769:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360655104 00000100:00100000:3.0:1788572573.680780:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360655104 00000100:00100000:3.0:1788572573.680828:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1064 00000100:00100000:3.0:1788572573.680837:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37265:x1875453360655104:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:3.0:1788572573.681319:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37265:x1875453360655104:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 481us (4182us total) trans 4294971004 rc 0/0 00000100:00100000:3.0:1788572573.681327:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1064 00000100:00100000:0.0:1788572573.685058:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.685305:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360655488 00000100:00100000:3.0:1788572573.685317:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360655488 00000100:00100000:3.0:1788572573.685404:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4907 00000100:00100000:3.0:1788572573.685416:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37269:x1875453360655488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572573.685585:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37269:x1875453360655488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 170us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.685602:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4907 00000100:00100000:2.0:1788572573.700542:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.701458:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360655616 00000100:00100000:3.0:1788572573.701476:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360655616 00002000:00100000:3.0:1788572573.701558:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5c:0x0] (0->1048575) req@ffff9627d0fb1180 x1875453360655616/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572573.701571:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d0fb1180 with x1875453360655616 ext(0->1048575) 00002000:00010000:3.0:1788572573.701578:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d0fb1180 00000100:00100000:3.0:1788572573.701631:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 368 00000100:00100000:3.0:1788572573.701644:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360655616:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1788572573.727811:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360655616:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 Request processed in 26149us (27265us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788572573.727830:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 368 00002000:00100000:3.0:1788572573.727859:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5c:0x0] (0->1048575) req@ffff9627d0fb1180 x1875453360655616/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/440 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572573.727871:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d0fb1180 with x1875453360655616 ext(0->1048575) 00002000:00010000:3.0:1788572573.727880:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d0fb1180 00000100:00100000:2.0:1788572573.728125:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.729251:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360655744 00000100:00100000:3.0:1788572573.729269:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360655744 00000100:00100000:3.0:1788572573.729359:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 409 00000100:00100000:3.0:1788572573.729369:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c97cf050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:2349:x1875453360655744:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572573.729463:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572573.729493:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165d8a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f170b expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729584:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729614:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729629:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729648:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729655:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729662:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729921:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729929:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729936:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729941:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aec00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572573.729946:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729954:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729969:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572573.729972:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572573.729977:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.729986:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb165d91 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730000:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730089:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730095:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730101:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572573.730103:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730109:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730116:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572573.730118:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572573.730121:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730126:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165d98 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730162:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165d8a lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f170b expref: 246 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730178:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572573.730181:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572573.730185:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165d8a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f170b expref: 246 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.730194:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5afc00, rc -2) 00000100:00100000:3.0:1788572573.730306:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c97cf050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:2349:x1875453360655744:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 936us (2181us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572573.730316:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 409 00000100:00100000:0.0:1788572573.737135:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.737291:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360655872 00000100:00100000:2.0:1788572573.737305:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360655872 00000100:00100000:2.0:1788572573.737389:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4908 00000100:00100000:2.0:1788572573.737402:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37270:x1875453360655872:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572573.737429:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572573.737457:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8400/0x85ace67feb165d9f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1743 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.737550:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.737568:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.737583:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572573.737639:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572573.737644:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572573.737668:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.737704:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8400/0x85ace67feb165d9f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 247 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.737727:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 246 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.737737:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572573.737740:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572573.737744:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b9800, rc 0) 00000100:00100000:2.0:1788572573.737825:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37270:x1875453360655872:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 426us (705us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.737833:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4908 00000100:00100000:1.0:1788572573.751422:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.751605:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360656128 00000100:00100000:2.0:1788572573.751620:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360656128 00000100:00100000:2.0:1788572573.751699:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4909 00000100:00100000:2.0:1788572573.751711:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37270:x1875453360656128:12345-192.168.202.40@tcp:58:lfs.0 00000040:00080000:2.0:1788572573.752324:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].170, 136 off31104 00000100:00100000:2.0:1788572573.752498:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37270:x1875453360656128:12345-192.168.202.40@tcp:58:lfs.0 Request processed in 787us (1090us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.752507:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4909 00000100:00100000:1.0:1788572573.757175:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.757356:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360656256 00000100:00100000:3.0:1788572573.757369:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360656256 00000100:00100000:3.0:1788572573.757442:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4910 00000100:00100000:3.0:1788572573.757454:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37269:x1875453360656256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572573.757481:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572573.757509:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165dad lrc: 2/0,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f174a expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.757553:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.757571:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.757595:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.757740:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165dad lrc: 0/0,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.757763:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.757774:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:3.0:1788572573.757777:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572573.757781:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af400, rc 0) 00000100:00100000:3.0:1788572573.757868:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37269:x1875453360656256:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 415us (706us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.757876:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4910 00000100:00100000:0.0:1788572573.766110:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.766394:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360656384 00000100:00100000:2.0:1788572573.766409:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360656384 00000100:00100000:2.0:1788572573.766489:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4911 00000100:00100000:2.0:1788572573.766500:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37269:x1875453360656384:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572573.766550:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xab:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572573.766704:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37269:x1875453360656384:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 205us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.766712:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4911 00000100:00100000:2.0:1788572573.773779:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.773919:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360656512 00000100:00100000:0.0:1788572573.773933:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360656512 00000100:00100000:0.0:1788572573.774072:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1430 00000100:00100000:0.0:1788572573.774086:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360656512:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572573.774146:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774170:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572573.774175:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572573.774183:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774201:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774242:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774252:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774261:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774269:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572573.774276:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774286:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774303:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.774312:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165dbb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.774593:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360656512:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 501us (818us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.774603:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1430 00000100:00100000:1.0:1788572573.782842:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.782979:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360656768 00000100:00100000:2.0:1788572573.782993:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360656768 00000100:00100000:2.0:1788572573.783131:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4912 00000100:00100000:2.0:1788572573.783143:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37269:x1875453360656768:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572573.783251:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.783271:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.783282:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.783328:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 63 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.783357:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 63 pid: 5838 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.783374:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382404096:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572573.783657:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56840 going to sleep for 11 seconds 00000100:00100000:2.0:1788572573.792604:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.794209:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360656896 00000100:00100000:1.0:1788572573.794231:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360656896 00010000:00010000:1.0:1788572573.794294:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 64 pid: 5838 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572573.794369:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1146 00000100:00100000:1.0:1788572573.794381:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360656896:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572573.794401:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572573.794409:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 64 pid: 5838 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.794425:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 64 pid: 5838 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.794454:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:1.0:1788572573.794461:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.794474:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.794483:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.794508:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572573.794599:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:2349:x1875453360656896:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 221us (2020us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.794607:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1146 00010000:00010000:3.0:1788572573.796261:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165db4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f174a expref: 63 pid: 5838 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.796289:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382404096:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572573.796327:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796351:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796358:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796630:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796640:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796652:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796659:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572573.796664:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796673:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796690:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:3.0:1788572573.796694:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572573.796699:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.796706:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165dc2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.796918:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37269:x1875453360656768:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 13774us (14088us total) trans 4294971005 rc 0/0 00000100:00100000:3.0:1788572573.796930:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4912 00000040:00080000:1.0:1788572573.797764:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1788572573.797773:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1788572573.797815:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 171 count 5 00000040:00080000:1.0:1788572573.797824:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1788572573.798192:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b52a00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382404224:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572573.798279:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c7f0f600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572573.802981:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.803190:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360657152 00000100:00100000:3.0:1788572573.803203:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360657152 00000100:00100000:3.0:1788572573.803269:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1065 00000100:00100000:3.0:1788572573.803281:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37269:x1875453360657152:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572573.803625:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37269:x1875453360657152:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 345us (658us total) trans 4294971006 rc 0/0 00000100:00100000:3.0:1788572573.803634:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1065 00000100:00100000:0.0:1788572573.805277:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.805315:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360657280 00000100:00100000:0.0:1788572573.805324:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360657280 00000100:00100000:0.0:1788572573.805407:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4913 00000100:00100000:0.0:1788572573.805419:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37277:x1875453360657280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572573.805453:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xac:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572573.805560:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37277:x1875453360657280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 143us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.805567:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4913 00000100:00100000:1.0:1788572573.805740:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b52a00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382404224:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572573.814170:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.814350:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360657408 00000100:00100000:2.0:1788572573.814365:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360657408 00000100:00100000:2.0:1788572573.814441:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4914 00000100:00100000:2.0:1788572573.814453:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37276:x1875453360657408:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:2.0:1788572573.814482:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572573.814509:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8200/0x85ace67feb165dc9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1751 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.814610:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8000/0x85ace67feb165dd0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.814629:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8000/0x85ace67feb165dd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.814644:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8000/0x85ace67feb165dd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572573.814690:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572573.814695:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572573.814719:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8000/0x85ace67feb165dd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.814751:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8200/0x85ace67feb165dc9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1751 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.814773:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8000/0x85ace67feb165dd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1751 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.814782:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572573.814786:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572573.814792:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b8000, rc 0) 00000100:00100000:2.0:1788572573.814881:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37276:x1875453360657408:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 429us (727us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.814889:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4914 00010000:00010000:0.0:1788572573.818323:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572573.824443:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.824673:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360657664 00000100:00100000:1.0:1788572573.824688:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360657664 00000100:00100000:1.0:1788572573.824771:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4915 00000100:00100000:1.0:1788572573.824784:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37277:x1875453360657664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572573.824818:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572573.824859:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165dd7 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1758 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.825054:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165dde lrc: 3/1,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.825081:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165dde lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.825097:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165dde lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572573.825111:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xac:0x0]: returning size 1048576 00010000:00010000:1.0:1788572573.825288:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165dd7 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1758 expref: 68 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.825308:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165dde lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1758 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.825317:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:1.0:1788572573.825322:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572573.825326:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd000, rc 0) 00000100:00100000:1.0:1788572573.825399:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37277:x1875453360657664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 618us (971us total) trans 4294971007 rc 0/0 00000100:00100000:1.0:1788572573.825407:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4915 00000100:00100000:3.0:1788572573.835872:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.835917:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.835983:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.836106:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360658048 00000100:00100000:1.0:1788572573.836124:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360658048 00000100:00100000:1.0:1788572573.836222:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 376 00000100:00100000:1.0:1788572573.836235:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2350:x1875453360658048:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572573.836257:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572573.836295:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb165dde lrc: 0/0,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1758 expref: 67 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.836322:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:1.0:1788572573.836330:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572573.836421:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2350:x1875453360658048:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 187us (436us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.836429:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 376 00000100:00100000:0.0:1788572573.836623:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360657792 00000100:00100000:0.0:1788572573.836629:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360657792 00000100:00100000:0.0:1788572573.836693:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4916 00000100:00100000:0.0:1788572573.836703:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37278:x1875453360657792:12345-192.168.202.40@tcp:101:rm.0 00010000:00010000:0.0:1788572573.836722:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.836745:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165dec lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f175f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.836824:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.836841:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.836854:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.836871:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.836878:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.836886:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837199:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837208:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837215:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837221:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.837227:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837240:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837252:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572573.837255:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.837260:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837268:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165df3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837279:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837285:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837292:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837297:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.837300:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837306:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837313:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627c5160500) 00010000:00010000:0.0:1788572573.837315:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.837318:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.837327:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165dfa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.837499:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360657920 00000100:00100000:2.0:1788572573.837505:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360657920 00000100:00100000:2.0:1788572573.837552:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1431 00000100:00100000:2.0:1788572573.837559:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360657920:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572573.837604:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837622:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.837626:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.837634:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837646:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837676:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837684:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837692:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837703:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b8200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572573.837707:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837714:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837724:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.837733:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e01 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.837794:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360657920:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 235us (1876us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.837801:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1431 00010000:00010000:0.0:1788572573.838162:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165dec lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f175f expref: 67 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.838173:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572573.838175:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.838180:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165dec lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f175f expref: 67 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.838189:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54fa00, rc 0) 00000100:00100000:0.0:1788572573.838214:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37278:x1875453360657792:12345-192.168.202.40@tcp:101:rm.0 Request processed in 1515us (2366us total) trans 0 rc 301/301 00000100:00100000:0.0:1788572573.838221:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4916 00000100:00100000:2.0:1788572573.844489:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.844599:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360658304 00000100:00100000:0.0:1788572573.844615:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360658304 00000100:00100000:0.0:1788572573.844718:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4917 00000100:00100000:0.0:1788572573.844733:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37277:x1875453360658304:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572573.844907:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37277:x1875453360658304:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 178us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.844916:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4917 00000100:00100000:3.0:1788572573.847839:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.847958:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360658432 00000100:00100000:1.0:1788572573.847971:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360658432 00000100:00100000:1.0:1788572573.848092:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1432 00000100:00100000:1.0:1788572573.848104:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2350:x1875453360658432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572573.848134:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572573.848167:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1766 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.848206:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572573.848212:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572573.848224:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 10 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.848245:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 10 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.848261:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bd000, rc 0) 00000100:00100000:1.0:1788572573.848331:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360658432:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 229us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.848338:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1432 00000100:00100000:3.0:1788572573.850709:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.851288:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360658688 00000100:00100000:1.0:1788572573.851305:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360658688 00000100:00100000:1.0:1788572573.851402:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4918 00000100:00100000:1.0:1788572573.851416:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37277:x1875453360658688:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572573.851591:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37277:x1875453360658688:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 168us (885us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.851600:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4918 00000100:00100000:3.0:1788572573.866711:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.866766:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.867044:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659072 00000100:00100000:2.0:1788572573.867070:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659072 00002000:00100000:2.0:1788572573.867146:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5c:0x0] (0->1048575) req@ffff9627ff5ec380 x1875453360659072/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.867161:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff5ec380 with x1875453360659072 ext(0->1048575) 00002000:00010000:2.0:1788572573.867169:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ff5ec380 00000100:00100000:2.0:1788572573.867210:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 369 00000100:00100000:2.0:1788572573.867223:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360659072:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1788572573.868439:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360658944 00000100:00100000:1.0:1788572573.868459:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360658944 00000100:00100000:1.0:1788572573.868543:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4919 00000100:00100000:1.0:1788572573.868555:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37279:x1875453360658944:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:1.0:1788572573.868597:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572573.868602:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788572573.868645:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8000/0x85ace67feb165dd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1751 expref: 66 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.868682:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572573.868690:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572573.868709:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165e0f lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f176d expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.869844:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.875791:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 245 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.875804:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.876608:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 245 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.876631:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 245 pid: 22536 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.876655:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382404352:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572573.876806:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572573.882506:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.882630:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659200 00000100:00100000:1.0:1788572573.882643:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659200 00010000:00010000:1.0:1788572573.882696:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 246 pid: 22536 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572573.882745:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1147 00000100:00100000:1.0:1788572573.882758:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:2349:x1875453360659200:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572573.882777:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572573.882786:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 246 pid: 22536 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.882801:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 246 pid: 22536 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.882835:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572573.882842:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.882865:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.882874:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572573.882900:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572573.882975:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:2349:x1875453360659200:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 218us (483us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.882983:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1147 00010000:00010000:0.0:1788572573.883904:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9800/0x85ace67feb165da6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1743 expref: 245 pid: 22536 timeout: 1021 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.883927:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596b100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382404352:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572573.883974:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.883985:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.883991:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.884073:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.884087:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.884097:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572573.885169:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5d:0x0], index 0: 0 00010000:00010000:0.0:1788572573.885473:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb165e24 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885493:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb165e24 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885505:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb165e24 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885553:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885562:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885568:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885574:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.885579:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885600:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885614:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572573.885617:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.885623:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885629:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc600/0x85ace67feb165e16 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572573.885642:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d46534c3 reply state 00000000327de860 transno 4294971008 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.885653:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788572573.885697:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165e0f lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f176d expref: 68 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885714:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb165e24 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f176d expref: 67 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.885722:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:0.0:1788572573.885725:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572573.885728:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e400, rc 0) 00000100:00100000:0.0:1788572573.885819:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37279:x1875453360658944:12345-192.168.202.40@tcp:101:dd.0 Request processed in 17266us (19124us total) trans 4294971008 rc 0/0 00000100:00100000:0.0:1788572573.885830:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4919 00010000:00010000:0.0:1788572573.892230:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.892254:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.892263:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.892269:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.892275:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.892285:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.892311:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627c5160500) 00010000:00010000:0.0:1788572573.892315:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572573.892321:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.892328:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e1d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.895038:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360659072:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 Request processed in 27771us (28225us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788572573.895062:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 369 00002000:00100000:2.0:1788572573.895081:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5c:0x0] (0->1048575) req@ffff9627ff5ec380 x1875453360659072/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/440 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.895093:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627ff5ec380 with x1875453360659072 ext(0->1048575) 00002000:00010000:2.0:1788572573.895101:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627ff5ec380 00000100:00100000:3.0:1788572573.925800:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.925925:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659328 00000100:00100000:0.0:1788572573.925938:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659328 00000100:00100000:0.0:1788572573.926079:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1433 00000100:00100000:0.0:1788572573.926094:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:37279:x1875453360659328:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572573.926121:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.926151:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1774 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572573.926187:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572573.926217:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=921s v=(0 0) 00010000:00010000:0.0:1788572573.926223:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572573.926236:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.926257:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 11 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.926273:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e800, rc 0) 00000100:00100000:0.0:1788572573.926529:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:37279:x1875453360659328:12345-192.168.202.40@tcp:101:dd.0 Request processed in 438us (743us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.926537:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1433 00000100:00100000:3.0:1788572573.937913:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.938161:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659456 00000100:00100000:0.0:1788572573.938173:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659456 00000100:00100000:0.0:1788572573.938261:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4920 00000100:00100000:0.0:1788572573.938272:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37277:x1875453360659456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572573.938303:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572573.938329:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165e39 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f177b expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.938556:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.938574:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.938602:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.938723:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165e39 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 69 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.938744:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.938754:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:0.0:1788572573.938757:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572573.938761:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:0.0:1788572573.938882:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37277:x1875453360659456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 611us (984us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572573.938891:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4920 00000100:00100000:3.0:1788572573.947874:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572573.948348:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659584 00000100:00100000:1.0:1788572573.948363:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659584 00000100:00100000:1.0:1788572573.948434:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4921 00000100:00100000:1.0:1788572573.948447:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37277:x1875453360659584:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572573.948493:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xac:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572573.948643:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37277:x1875453360659584:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 198us (784us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572573.948652:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4921 00000100:00100000:3.0:1788572573.955292:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.955478:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659712 00000100:00100000:2.0:1788572573.955492:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659712 00002000:00100000:2.0:1788572573.955565:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5d:0x0] (0->1048575) req@ffff9627e786c000 x1875453360659712/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/0 e 0 to 0 dl 1788572584 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.955577:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786c000 with x1875453360659712 ext(0->1048575) 00002000:00010000:2.0:1788572573.955598:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786c000 00000100:00100000:2.0:1788572573.955647:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 370 00000100:00100000:2.0:1788572573.955662:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360659712:12345-192.168.202.40@tcp:4:dd.0 00000100:00100000:0.0:1788572573.959993:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.960617:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360659840 00000100:00100000:2.0:1788572573.960634:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360659840 00000100:00100000:2.0:1788572573.960734:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1434 00000100:00100000:2.0:1788572573.960750:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360659840:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572573.960816:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960838:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=1s v=(1 0) 00010000:00010000:2.0:1788572573.960845:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572573.960853:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960875:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960916:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960927:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960936:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960944:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b8200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572573.960949:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960959:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960979:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.960988:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004117048c/0x85ace67feb165e47 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.961147:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360659840:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 400us (1168us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572573.961156:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1434 00000100:00100000:0.0:1788572573.966844:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572573.966947:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360660096 00000100:00100000:2.0:1788572573.966961:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360660096 00000100:00100000:2.0:1788572573.967067:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4922 00000100:00100000:2.0:1788572573.967090:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37277:x1875453360660096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572573.967199:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.967220:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.967232:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 4/0,1 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.967283:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572573.967300:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 68 pid: 22536 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572573.967317:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b31880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382404480:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572573.967434:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572573.978997:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b31880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382404480:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572573.979094:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572573.980728:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572573.981312:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360660224 00000100:00100000:3.0:1788572573.981336:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360660224 00010000:00010000:3.0:1788572573.981390:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 69 pid: 22536 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572573.981441:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1148 00000100:00100000:3.0:1788572573.981453:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2350:x1875453360660224:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572573.981472:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572573.981481:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 69 pid: 22536 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.981497:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 69 pid: 22536 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.981528:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165e40 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f177b expref: 69 pid: 22536 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.981550:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:3.0:1788572573.981557:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.981570:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.981590:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572573.981630:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572573.981703:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2350:x1875453360660224:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 250us (984us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572573.981712:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1148 00010000:00010000:0.0:1788572573.981843:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.981862:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982163:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982172:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982180:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982186:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572573.982192:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982202:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982214:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:0.0:1788572573.982217:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572573.982221:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 2/0,0 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572573.982228:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9e00/0x85ace67feb165e4e lrc: 0/0,0 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572573.982389:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37277:x1875453360660096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 15309us (15556us total) trans 4294971009 rc 0/0 00000100:00100000:0.0:1788572573.982400:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4922 00000040:00080000:3.0:1788572573.983308:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572573.983316:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572573.983819:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:3.0:1788572573.983829:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572573.986328:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572573.986422:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360660480 00000100:00100000:0.0:1788572573.986435:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360660480 00000100:00100000:0.0:1788572573.986526:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1066 00000100:00100000:0.0:1788572573.986540:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37277:x1875453360660480:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572573.986958:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37277:x1875453360660480:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 420us (640us total) trans 4294971010 rc 0/0 00000100:00100000:0.0:1788572573.986968:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1066 00000100:00100000:2.0:1788572573.996617:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360659712:12345-192.168.202.40@tcp:4:dd.0 Request processed in 40941us (41324us total) trans 4294967674 rc 0/0 00000100:00100000:2.0:1788572573.996639:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 370 00002000:00100000:2.0:1788572573.996656:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5d:0x0] (0->1048575) req@ffff9627e786c000 x1875453360659712/t4294967674(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:234/0 lens 488/448 e 0 to 0 dl 1788572584 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572573.996667:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e786c000 with x1875453360659712 ext(0->1048575) 00002000:00010000:2.0:1788572573.996674:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627e786c000 00000100:00100000:3.0:1788572574.000447:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.001428:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360660608 00000100:00100000:2.0:1788572574.001443:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360660608 00000100:00100000:2.0:1788572574.001534:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4923 00000100:00080000:1.0:1788572574.001549:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572574.001549:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37279:x1875453360660608:12345-192.168.202.40@tcp:44:dd.0 00000100:00100000:3.0:1788572574.015983:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b52a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382404608:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788572574.016172:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572574.017047:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382404608 00000100:00100000:2.0:1788572574.017067:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382404608 00000100:00100000:2.0:1788572574.017152:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:2.0:1788572574.017163:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382404608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788572574.017196:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 197197824 pending 0 free 3855613952 avail 3852468224 00000100:00100000:2.0:1788572574.018423:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382404608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1256us (2246us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.018435:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1788572574.019276:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b52a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382404608:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1788572574.061865:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971010 is committed 00000001:00080000:3.0:1788572574.061922:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971004 is committed 00000100:00100000:0.0:1788572574.062256:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37279:x1875453360660608:12345-192.168.202.40@tcp:44:dd.0 Request processed in 60703us (61812us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.062270:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4923 00000100:00100000:0.0:1788572574.066130:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.066279:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360660736 00000100:00100000:1.0:1788572574.066293:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360660736 00000100:00100000:1.0:1788572574.066795:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1435 00000100:00100000:1.0:1788572574.066808:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360660736:12345-192.168.202.40@tcp:16:dd.0 00000001:00080000:1.0:1788572574.102724:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967674 is committed 00000100:00100000:0.0:1788572574.103131:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360660736:12345-192.168.202.40@tcp:16:dd.0 Request processed in 36316us (37007us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.103147:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1435 00000100:00100000:2.0:1788572574.105639:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.105766:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360660992 00000100:00100000:1.0:1788572574.105779:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360660992 00000100:00100000:1.0:1788572574.105938:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1067 00000100:00100000:1.0:1788572574.105952:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec039850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37279:x1875453360660992:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:1.0:1788572574.106524:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec039850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37279:x1875453360660992:12345-192.168.202.40@tcp:35:dd.0 Request processed in 572us (895us total) trans 4294971011 rc 0/0 00000100:00100000:1.0:1788572574.112283:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1067 00000100:00100000:2.0:1788572574.156460:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.156562:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360661248 00000100:00100000:1.0:1788572574.156583:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360661248 00000100:00100000:1.0:1788572574.156661:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4924 00000100:00100000:1.0:1788572574.156674:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37283:x1875453360661248:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:1.0:1788572574.156705:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572574.156732:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165e55 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1782 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.156833:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.156984:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.156998:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157056:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157063:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157068:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157446:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157458:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157466:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572574.157480:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xad:0x0]: returning size 1048576 00010000:00010000:1.0:1788572574.157616:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157625:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157633:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157639:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.157644:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157657:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157672:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.157676:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.157682:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157690:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165e5c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157704:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157710:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157716:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157721:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.157724:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157729:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157737:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627ff5ab600) 00010000:00010000:1.0:1788572574.157739:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.157742:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157749:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb165e63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157804:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb165e55 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 246 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157827:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 245 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.157834:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:1.0:1788572574.157837:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572574.157840:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5afc00, rc 0) 00000100:00100000:1.0:1788572574.159844:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37283:x1875453360661248:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 3157us (3381us total) trans 4294971012 rc 0/0 00000100:00100000:1.0:1788572574.159861:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4924 00000100:00100000:0.0:1788572574.163138:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572574.163271:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360661376 00000100:00100000:3.0:1788572574.163285:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360661376 00000100:00100000:3.0:1788572574.163352:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1436 00000100:00100000:3.0:1788572574.163365:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2347:x1875453360661376:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572574.163391:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572574.163412:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000198c5525/0x85ace67feb165e78 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1789 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572574.163484:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382404736:192.168.202.40@tcp:106: 00000100:00100000:3.0:1788572574.163608:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c63ccf00 going to sleep for 11 seconds 00002000:00010000:2.0:1788572574.168708:0:13272:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x5d:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:2.0:1788572574.168722:0:13272:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x5d:0x0] updating lvb mtime: 0 -> 1788572573 00002000:00010000:2.0:1788572574.168726:0:13272:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x5d:0x0] updating lvb atime: 0 -> 1788572573 00002000:00010000:2.0:1788572574.168729:0:13272:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x5d:0x0] updating lvb ctime: 0 -> 1788572573 00002000:00010000:2.0:1788572574.168731:0:13272:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x5d:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:3.0:1788572574.168858:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382404736:192.168.202.40@tcp:106: 00010000:00010000:3.0:1788572574.168970:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000198c5525/0x85ace67feb165e78 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1789 expref: 73 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.169216:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000198c5525/0x85ace67feb165e78 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1789 expref: 73 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.169235:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e7a97c00, rc 0) 00000100:00100000:3.0:1788572574.169376:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2347:x1875453360661376:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 6011us (6253us total) trans 0 rc 301/301 00000100:00100000:3.0:1788572574.169388:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1436 00000100:00100000:1.0:1788572574.173064:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.173684:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360661504 00000100:00100000:2.0:1788572574.173700:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360661504 00000100:00100000:2.0:1788572574.173828:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1437 00000100:00100000:2.0:1788572574.173841:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:37283:x1875453360661504:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572574.173876:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572574.173901:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1790 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.173942:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 10 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.173951:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x5d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1788572574.173956:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572574.173962:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 4/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.174061:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 10 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.174078:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 10 pid: 6679 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572574.174095:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b32d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382404864:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572574.174205:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572574.178580:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b32d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382404864:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572574.178647:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 3/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.178681:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b9a00, rc 0) 00000100:00100000:2.0:1788572574.178808:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:37283:x1875453360661504:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 4969us (5819us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.178819:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1437 00000100:00100000:3.0:1788572574.181164:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.183204:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360661632 00000100:00100000:0.0:1788572574.183225:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360661632 00010000:00010000:0.0:1788572574.183312:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 11 pid: 6679 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572574.183539:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1149 00000100:00100000:0.0:1788572574.183554:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360661632:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572574.183586:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572574.183617:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 11 pid: 6679 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.183634:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 11 pid: 6679 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.183674:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071068e57/0x85ace67feb165e32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1774 expref: 11 pid: 6679 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.183695:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572574.183700:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572574.183710:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.183720:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.183770:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 72 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572574.183786:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382404992:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572574.183917:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572574.187742:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.188312:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360661760 00000100:00100000:1.0:1788572574.188334:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360661760 00002000:00100000:1.0:1788572574.188397:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5d:0x0] (0->32767) req@ffff9626c1d8bb80 x1875453360661760/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/0 e 0 to 0 dl 1788572585 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572574.188411:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1d8bb80 with x1875453360661760 ext(0->32767) 00002000:00010000:1.0:1788572574.188418:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1d8bb80 00000100:00100000:1.0:1788572574.188451:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 371 00000100:00100000:1.0:1788572574.188461:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360661760:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:0.0:1788572574.191373:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382404992:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572574.191434:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572574.191585:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360661632:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8033us (10435us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.191599:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1149 00000100:00100000:0.0:1788572574.192974:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360661760:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 4503us (5233us total) trans 0 rc 32768/32768 00000100:00100000:0.0:1788572574.192990:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 371 00002000:00100000:0.0:1788572574.193037:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5d:0x0] (0->32767) req@ffff9626c1d8bb80 x1875453360661760/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/440 e 0 to 0 dl 1788572585 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572574.193051:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1d8bb80 with x1875453360661760 ext(0->32767) 00002000:00010000:0.0:1788572574.193059:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1d8bb80 00000100:00100000:2.0:1788572574.199063:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.199129:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360661888 00000100:00100000:2.0:1788572574.199146:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360661888 00002000:00100000:2.0:1788572574.199231:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5d:0x0] (32768->1048575) req@ffff9626c4b32d80 x1875453360661888/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/0 e 0 to 0 dl 1788572585 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572574.199246:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b32d80 with x1875453360661888 ext(32768->1048575) 00002000:00010000:2.0:1788572574.199254:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b32d80 00000100:00100000:2.0:1788572574.199299:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 372 00000100:00100000:2.0:1788572574.199312:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360661888:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572574.218911:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360661888:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 19586us (19850us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572574.218929:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 372 00002000:00100000:2.0:1788572574.218945:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5d:0x0] (32768->1048575) req@ffff9626c4b32d80 x1875453360661888/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/440 e 0 to 0 dl 1788572585 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572574.218955:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b32d80 with x1875453360661888 ext(32768->1048575) 00002000:00010000:2.0:1788572574.218963:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b32d80 00000100:00100000:2.0:1788572574.227756:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572574.227877:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360662144 00000100:00100000:3.0:1788572574.227892:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360662144 00000100:00100000:3.0:1788572574.227969:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1068 00000100:00100000:3.0:1788572574.227981:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37283:x1875453360662144:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:3.0:1788572574.228382:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37283:x1875453360662144:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 402us (640us total) trans 4294971013 rc 0/0 00000100:00100000:3.0:1788572574.228390:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1068 00000100:00100000:3.0:1788572574.245630:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.245725:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360662272 00000100:00100000:0.0:1788572574.245744:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360662272 00000100:00100000:0.0:1788572574.245864:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 410 00000100:00100000:0.0:1788572574.245885:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:2347:x1875453360662272:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572574.245948:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.245976:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e86 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1797 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246119:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246139:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246156:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246176:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246184:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246191:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246503:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246512:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246518:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246525:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572574.246530:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246538:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246553:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.246557:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572574.246563:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246579:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165e8d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246593:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246599:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246604:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246609:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572574.246612:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246618:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246625:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xe5a305b6 (ffff9627c5160100) 00010000:00010000:0.0:1788572574.246628:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572574.246631:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246637:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165e94 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246678:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e86 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1797 expref: 247 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246689:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.246691:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572574.246694:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e86 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1797 expref: 247 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.246709:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54e800, rc -2) 00000100:00100000:0.0:1788572574.246815:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:2347:x1875453360662272:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 937us (1194us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572574.246824:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 410 00000100:00100000:2.0:1788572574.251348:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.251819:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360662400 00000100:00100000:0.0:1788572574.251835:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360662400 00000100:00100000:0.0:1788572574.251944:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4925 00000100:00100000:0.0:1788572574.251959:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37285:x1875453360662400:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572574.251989:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.252100:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e9b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f179e expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.252192:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.252210:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.252225:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572574.252271:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572574.252276:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572574.252297:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.252678:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165e9b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 248 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.252708:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 247 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.252718:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.252722:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572574.252727:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f600, rc 0) 00000100:00100000:0.0:1788572574.252816:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37285:x1875453360662400:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 860us (1482us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.252825:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4925 00000100:00100000:2.0:1788572574.258127:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.258285:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360662656 00000100:00100000:0.0:1788572574.258299:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360662656 00000100:00100000:0.0:1788572574.258404:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4926 00000100:00100000:0.0:1788572574.258418:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37285:x1875453360662656:12345-192.168.202.40@tcp:58:lfs.0 00000040:00080000:0.0:1788572574.259080:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].171, 136 off31240 00000100:00100000:0.0:1788572574.259203:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37285:x1875453360662656:12345-192.168.202.40@tcp:58:lfs.0 Request processed in 787us (1090us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.259213:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4926 00000100:00100000:3.0:1788572574.291974:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.292050:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360662784 00000100:00100000:2.0:1788572574.292065:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360662784 00000100:00100000:2.0:1788572574.292145:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4927 00000100:00100000:2.0:1788572574.292158:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37289:x1875453360662784:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:2.0:1788572574.292191:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572574.292222:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8a00/0x85ace67feb165ea9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17a5 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.292319:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb165eb0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.292508:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb165eb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.292525:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb165eb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572574.292589:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572574.292594:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572574.292622:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb165eb0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.292657:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8a00/0x85ace67feb165ea9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17a5 expref: 64 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.292686:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb165eb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17a5 expref: 63 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.292697:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572574.292700:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572574.292706:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b9c00, rc 0) 00000100:00100000:2.0:1788572574.292844:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37289:x1875453360662784:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 687us (885us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.292862:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4927 00000100:00100000:3.0:1788572574.303430:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.304245:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360662912 00000100:00100000:1.0:1788572574.304261:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360662912 00000100:00100000:1.0:1788572574.304468:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4928 00000100:00100000:1.0:1788572574.304482:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37290:x1875453360662912:12345-192.168.202.40@tcp:101:rm.0 00010000:00010000:1.0:1788572574.304510:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572574.304539:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165eb7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17ac expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.304646:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.304665:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.304679:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.304695:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.304702:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.304708:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305474:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305488:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305495:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305503:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.305508:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305517:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305535:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.305538:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.305545:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305551:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb165ebe lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305575:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305581:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305586:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305591:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.305594:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305599:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305612:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627ff5ab600) 00010000:00010000:1.0:1788572574.305614:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.305617:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305623:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb165ec5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305673:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165eb7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ac expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305685:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.305687:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.305691:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165eb7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ac expref: 64 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.305703:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af800, rc 0) 00000100:00100000:1.0:1788572574.305799:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37290:x1875453360662912:12345-192.168.202.40@tcp:101:rm.0 Request processed in 1319us (2383us total) trans 0 rc 301/301 00000100:00100000:1.0:1788572574.305809:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4928 00000100:00100000:2.0:1788572574.318164:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.318434:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360663168 00000100:00100000:0.0:1788572574.318447:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360663168 00000100:00100000:0.0:1788572574.318540:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4929 00000100:00100000:0.0:1788572574.318552:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37291:x1875453360663168:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572574.318587:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.318591:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572574.318625:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb165eb0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17a5 expref: 63 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.318657:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.318664:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572574.318682:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165ecc lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17b3 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.319513:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.319534:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 246 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.319544:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.319597:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 246 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.319613:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 246 pid: 5838 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572574.319629:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382405120:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572574.319817:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982d180 going to sleep for 11 seconds 00000100:00100000:3.0:1788572574.324940:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.325518:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360663296 00000100:00100000:2.0:1788572574.325535:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360663296 00010000:00010000:2.0:1788572574.325616:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 247 pid: 5838 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572574.325670:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1150 00000100:00100000:2.0:1788572574.325683:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2349:x1875453360663296:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572574.325703:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572574.325712:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 247 pid: 5838 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.325728:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 247 pid: 5838 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.325756:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572574.325762:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.325775:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.325785:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.325815:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572574.325890:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2349:x1875453360663296:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 209us (965us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.325898:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1150 00010000:00010000:0.0:1788572574.329522:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ea2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f179e expref: 246 pid: 5838 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572574.329557:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382405120:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572574.329625:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.329640:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.329646:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.329689:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.329707:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.329719:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572574.332624:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x5d:0x0], index 1: 0 00010000:00010000:1.0:1788572574.333466:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165ee1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333494:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333511:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333575:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333583:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333589:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333595:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54e200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.333600:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333608:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333623:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.333626:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.333632:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333640:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e200/0x85ace67feb165ed3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572574.333656:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008d15d5c7 reply state 00000000df9f3bfb transno 4294971014 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.333667:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1788572574.333716:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e800/0x85ace67feb165ecc lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17b3 expref: 65 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333740:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165ee1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17b3 expref: 64 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.333748:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:1.0:1788572574.333751:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572574.333753:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af800, rc 0) 00000100:00100000:1.0:1788572574.333929:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37291:x1875453360663168:12345-192.168.202.40@tcp:101:dd.0 Request processed in 15378us (15776us total) trans 4294971014 rc 0/0 00000100:00100000:1.0:1788572574.333940:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4929 00010000:00010000:2.0:1788572574.346306:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.347218:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.347228:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.347235:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fa00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572574.347241:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.347251:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.347275:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627c5160100) 00010000:00010000:2.0:1788572574.347279:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572574.347285:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.347293:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fa00/0x85ace67feb165eda lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.358452:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.359165:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360663424 00000100:00100000:2.0:1788572574.359181:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360663424 00000100:00100000:2.0:1788572574.359251:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1438 00000100:00100000:2.0:1788572574.359264:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:37291:x1875453360663424:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:2.0:1788572574.359287:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572574.359326:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17ba expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572574.359361:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1788572574.359390:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=922s v=(0 0) 00010000:00010000:2.0:1788572574.359396:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572574.359408:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.359428:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 11 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.359443:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b8a00, rc 0) 00000100:00100000:2.0:1788572574.359525:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:37291:x1875453360663424:12345-192.168.202.40@tcp:101:dd.0 Request processed in 264us (1090us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.359532:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1438 00000100:00100000:3.0:1788572574.378342:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.378787:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360663552 00000100:00100000:2.0:1788572574.378801:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360663552 00002000:00100000:2.0:1788572574.378870:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5d:0x0] (0->1048575) req@ffff9626c13bf800 x1875453360663552/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:235/0 lens 488/0 e 0 to 0 dl 1788572585 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572574.378883:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bf800 with x1875453360663552 ext(0->1048575) 00002000:00010000:2.0:1788572574.378890:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bf800 00000100:00100000:2.0:1788572574.378930:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 373 00000100:00100000:2.0:1788572574.378943:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360663552:12345-192.168.202.40@tcp:4:dd.0 00000100:00100000:0.0:1788572574.406897:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360663552:12345-192.168.202.40@tcp:4:dd.0 Request processed in 27932us (28555us total) trans 4294967675 rc 0/0 00000100:00100000:0.0:1788572574.406921:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 373 00002000:00100000:0.0:1788572574.406939:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5d:0x0] (0->1048575) req@ffff9626c13bf800 x1875453360663552/t4294967675(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:235/0 lens 488/448 e 0 to 0 dl 1788572585 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572574.406953:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c13bf800 with x1875453360663552 ext(0->1048575) 00002000:00010000:0.0:1788572574.406963:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c13bf800 00000100:00100000:3.0:1788572574.410716:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.410819:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360663680 00000100:00100000:1.0:1788572574.410834:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360663680 00000100:00100000:1.0:1788572574.410953:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4930 00000100:00100000:1.0:1788572574.410965:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37291:x1875453360663680:12345-192.168.202.40@tcp:44:dd.0 00000001:00080000:0.0:1788572574.460572:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971014 is committed 00000001:00080000:0.0:1788572574.460603:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971012 is committed 00000001:00080000:0.0:1788572574.460613:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971013 is committed 00000100:00100000:1.0:1788572574.460829:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37291:x1875453360663680:12345-192.168.202.40@tcp:44:dd.0 Request processed in 49859us (50122us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572574.460843:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4930 00000100:00100000:1.0:1788572574.463698:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.464758:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360663808 00000100:00100000:0.0:1788572574.464777:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360663808 00000100:00100000:0.0:1788572574.464916:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1439 00000100:00100000:0.0:1788572574.464931:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360663808:12345-192.168.202.40@tcp:16:dd.0 00000001:00080000:3.0:1788572574.521757:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967675 is committed 00000100:00100000:0.0:1788572574.522107:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360663808:12345-192.168.202.40@tcp:16:dd.0 Request processed in 57163us (58408us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.522123:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1439 00000100:00100000:3.0:1788572574.525289:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.525954:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664064 00000100:00100000:1.0:1788572574.525969:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664064 00000100:00100000:1.0:1788572574.526480:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1069 00000100:00100000:1.0:1788572574.526498:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37291:x1875453360664064:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:1.0:1788572574.527617:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37291:x1875453360664064:12345-192.168.202.40@tcp:35:dd.0 Request processed in 1115us (2334us total) trans 4294971015 rc 0/0 00000100:00100000:1.0:1788572574.527635:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1069 00000100:00100000:3.0:1788572574.595195:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.595374:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664320 00000100:00100000:0.0:1788572574.595388:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664320 00000100:00100000:0.0:1788572574.595464:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4931 00000100:00100000:0.0:1788572574.595477:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37295:x1875453360664320:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572574.595506:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.595536:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ef6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17c1 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.595652:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.595671:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.595685:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.595703:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.595710:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.595717:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596219:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 3/1,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596231:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596239:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572574.596250:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xae:0x0]: returning size 1048576 00010000:00010000:0.0:1788572574.596370:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596378:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596385:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596392:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572574.596397:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596406:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596422:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.596426:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572574.596432:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596444:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165efd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596458:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596464:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596470:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596475:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572574.596478:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596485:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596492:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627c5160100) 00010000:00010000:0.0:1788572574.596494:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572574.596497:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596502:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f04 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596568:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165ef6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 247 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596590:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 246 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.596598:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:0.0:1788572574.596600:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572574.596603:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f200, rc 0) 00000100:00100000:0.0:1788572574.596692:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37295:x1875453360664320:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 1217us (1524us total) trans 4294971016 rc 0/0 00000100:00100000:0.0:1788572574.596702:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4931 00000100:00100000:3.0:1788572574.603578:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.604185:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664448 00000100:00100000:1.0:1788572574.604199:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664448 00000100:00100000:1.0:1788572574.604279:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1440 00000100:00100000:1.0:1788572574.604291:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360664448:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:1.0:1788572574.604327:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572574.604359:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d91e1db3/0x85ace67feb165f19 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17c8 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.604438:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382405248:192.168.202.40@tcp:106: 00000100:00100000:1.0:1788572574.604577:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8e40 going to sleep for 11 seconds 00002000:00010000:1.0:1788572574.611349:0:20911:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x5d:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:1.0:1788572574.611366:0:20911:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x5d:0x0] updating lvb mtime: 0 -> 1788572573 00002000:00010000:1.0:1788572574.611369:0:20911:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x5d:0x0] updating lvb atime: 0 -> 1788572573 00002000:00010000:1.0:1788572574.611372:0:20911:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x5d:0x0] updating lvb ctime: 0 -> 1788572573 00002000:00010000:1.0:1788572574.611375:0:20911:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x5d:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:1.0:1788572574.611408:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382405248:192.168.202.40@tcp:106: 00010000:00010000:1.0:1788572574.611466:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d91e1db3/0x85ace67feb165f19 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c8 expref: 85 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.611501:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d91e1db3/0x85ace67feb165f19 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c8 expref: 85 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.611519:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aec00, rc 0) 00000100:00100000:1.0:1788572574.611612:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360664448:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 7321us (8057us total) trans 0 rc 301/301 00000100:00100000:1.0:1788572574.611624:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1440 00000100:00100000:3.0:1788572574.618336:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.618448:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664576 00000100:00100000:2.0:1788572574.618465:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664576 00000100:00100000:2.0:1788572574.618560:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1441 00000100:00100000:2.0:1788572574.618574:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:37295:x1875453360664576:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572574.618602:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572574.618629:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17cf expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.618675:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 10 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.618686:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572574.618691:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572574.618697:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 85 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.618745:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 10 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.618761:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 10 pid: 10416 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572574.618777:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382405376:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572574.618895:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572574.630275:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382405376:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572574.630355:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 85 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.630384:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b9200, rc 0) 00000100:00100000:2.0:1788572574.630485:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:37295:x1875453360664576:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 11913us (12167us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.630495:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1441 00000100:00100000:3.0:1788572574.634449:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.635048:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664704 00000100:00100000:1.0:1788572574.635063:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664704 00010000:00010000:1.0:1788572574.635120:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 11 pid: 10416 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.635181:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1151 00000100:00100000:1.0:1788572574.635194:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360664704:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572574.635215:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572574.635244:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 11 pid: 10416 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.635262:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 11 pid: 10416 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.635289:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000048acb834/0x85ace67feb165eef lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17ba expref: 11 pid: 10416 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.635310:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572574.635315:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572574.635327:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 84 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.635339:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 84 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.635399:0:23057:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 84 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.635416:0:23057:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382405504:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572574.635548:0:23057:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572574.643207:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.643374:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664832 00000100:00100000:0.0:1788572574.643388:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664832 00002000:00100000:0.0:1788572574.643459:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5d:0x0] (0->32767) req@ffff9627c121ed80 x1875453360664832/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/0 e 0 to 0 dl 1788572585 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572574.643472:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c121ed80 with x1875453360664832 ext(0->32767) 00002000:00010000:0.0:1788572574.643479:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c121ed80 00000100:00100000:0.0:1788572574.643579:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 374 00000100:00100000:0.0:1788572574.643592:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2350:x1875453360664832:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:1.0:1788572574.643989:0:23057:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:23057:1875453382405504:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572574.644045:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572574.644100:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360664704:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 8898us (9657us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572574.644111:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1151 00000100:00100000:0.0:1788572574.647966:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2350:x1875453360664832:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 4359us (4756us total) trans 0 rc 32768/32768 00000100:00100000:0.0:1788572574.647985:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 374 00002000:00100000:0.0:1788572574.648001:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5d:0x0] (0->32767) req@ffff9627c121ed80 x1875453360664832/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/440 e 0 to 0 dl 1788572585 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572574.648047:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c121ed80 with x1875453360664832 ext(0->32767) 00002000:00010000:0.0:1788572574.648055:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c121ed80 00000100:00100000:3.0:1788572574.655447:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.655673:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360664960 00000100:00100000:2.0:1788572574.655687:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360664960 00002000:00100000:2.0:1788572574.655753:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5d:0x0] (32768->1048575) req@ffff9627c121d880 x1875453360664960/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/0 e 0 to 0 dl 1788572585 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572574.655765:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c121d880 with x1875453360664960 ext(32768->1048575) 00002000:00010000:2.0:1788572574.655771:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c121d880 00000100:00100000:2.0:1788572574.655852:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 375 00000100:00100000:2.0:1788572574.655866:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360664960:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572574.670447:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360664960:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 14569us (15003us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572574.670463:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 375 00002000:00100000:2.0:1788572574.670480:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5d:0x0] (32768->1048575) req@ffff9627c121d880 x1875453360664960/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:235/0 lens 488/440 e 0 to 0 dl 1788572585 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572574.670491:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c121d880 with x1875453360664960 ext(32768->1048575) 00002000:00010000:2.0:1788572574.670499:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c121d880 00000100:00100000:3.0:1788572574.680704:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.680813:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360665216 00000100:00100000:0.0:1788572574.680826:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360665216 00000100:00100000:0.0:1788572574.680911:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1070 00000100:00100000:0.0:1788572574.680925:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37295:x1875453360665216:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:0.0:1788572574.681539:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37295:x1875453360665216:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 614us (847us total) trans 4294971017 rc 0/0 00000100:00100000:0.0:1788572574.681559:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1070 00000100:00100000:1.0:1788572574.711272:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.711747:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360665344 00000100:00100000:2.0:1788572574.711764:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360665344 00000100:00100000:2.0:1788572574.712102:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 411 00000100:00100000:2.0:1788572574.712115:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2347:x1875453360665344:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:2.0:1788572574.712174:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572574.712202:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9600/0x85ace67feb165f27 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17d6 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712276:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712295:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712320:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712337:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712345:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712351:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712630:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712639:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712646:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712656:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b8600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572574.712661:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712670:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712685:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572574.712688:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572574.712693:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712700:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb165f2e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712714:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712720:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712726:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712732:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b8800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572574.712734:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712741:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712753:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xe5a305b6 (ffff9627e3306900) 00010000:00010000:2.0:1788572574.712755:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572574.712758:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712765:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb165f35 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xe5a305b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712804:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9600/0x85ace67feb165f27 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17d6 expref: 248 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712814:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572574.712816:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572574.712820:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9600/0x85ace67feb165f27 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17d6 expref: 248 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572574.712830:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b9600, rc -2) 00000100:00100000:2.0:1788572574.713412:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2347:x1875453360665344:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1296us (2152us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572574.713423:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 411 00000100:00100000:1.0:1788572574.715503:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.715602:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360665472 00000100:00100000:1.0:1788572574.715615:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360665472 00000100:00100000:1.0:1788572574.715687:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4932 00000100:00100000:1.0:1788572574.715699:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37297:x1875453360665472:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572574.715727:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572574.715754:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f3c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17dd expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.716613:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.716637:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.716657:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572574.716708:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:1.0:1788572574.716712:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572574.716735:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.716772:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f3c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 249 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.716795:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.716805:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.716807:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572574.716812:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3336800, rc 0) 00000100:00100000:1.0:1788572574.717206:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37297:x1875453360665472:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1508us (1718us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572574.717216:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4932 00000100:00100000:1.0:1788572574.721234:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572574.721415:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360665728 00000100:00100000:2.0:1788572574.721429:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360665728 00000100:00100000:2.0:1788572574.721504:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4933 00000100:00100000:2.0:1788572574.721516:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37297:x1875453360665728:12345-192.168.202.40@tcp:58:lfs.0 00000040:00080000:2.0:1788572574.722377:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].172, 136 off31376 00000100:00100000:2.0:1788572574.722509:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37297:x1875453360665728:12345-192.168.202.40@tcp:58:lfs.0 Request processed in 991us (1286us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.722525:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4933 00000100:00100000:3.0:1788572574.756790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.757288:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360665856 00000100:00100000:0.0:1788572574.757311:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360665856 00000100:00100000:0.0:1788572574.757384:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4934 00000100:00100000:0.0:1788572574.757396:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37301:x1875453360665856:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:0.0:1788572574.757430:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.757458:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165f4a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17e4 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.757559:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f51 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.757577:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.757590:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572574.757636:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572574.757640:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572574.757662:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.757697:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165f4a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17e4 expref: 65 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.757722:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17e4 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.757733:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.757736:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572574.757741:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54f800, rc 0) 00000100:00100000:0.0:1788572574.757827:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37301:x1875453360665856:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 433us (1053us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.757835:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4934 00000100:00100000:3.0:1788572574.779750:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572574.780424:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360665984 00000100:00100000:1.0:1788572574.780441:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360665984 00000100:00100000:1.0:1788572574.780536:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4935 00000100:00100000:1.0:1788572574.780582:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37302:x1875453360665984:12345-192.168.202.40@tcp:101:rm.0 00010000:00010000:1.0:1788572574.780615:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572574.780648:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f58 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17eb expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.780748:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.780769:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.780785:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.780824:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.780833:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.780840:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.782441:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.782465:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.782475:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.782734:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3336400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.782740:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.782754:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.782774:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.782778:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.782785:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783306:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165f5f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783572:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783580:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783587:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783592:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572574.783596:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783602:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783614:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627ff5abe00) 00010000:00010000:1.0:1788572574.783617:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.783621:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783632:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165f66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783705:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f58 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17eb expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783719:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.783722:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572574.783727:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f58 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17eb expref: 65 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.783738:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3337600, rc 0) 00000100:00100000:1.0:1788572574.784569:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37302:x1875453360665984:12345-192.168.202.40@tcp:101:rm.0 Request processed in 4016us (4830us total) trans 0 rc 301/301 00000100:00100000:1.0:1788572574.784581:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4935 00000100:00100000:3.0:1788572574.812589:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.812774:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360666240 00000100:00100000:0.0:1788572574.812791:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360666240 00000100:00100000:0.0:1788572574.812894:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4936 00000100:00100000:0.0:1788572574.812909:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37303:x1875453360666240:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572574.812944:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.812948:0:9040:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788572574.812987:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f800/0x85ace67feb165f51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17e4 expref: 64 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.813056:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572574.813064:0:9040:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572574.813357:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165f6d lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17f2 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.813738:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.813755:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 247 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.813766:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.813807:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 247 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.813842:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 247 pid: 5837 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572574.813866:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382405632:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572574.813989:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572574.820920:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.821394:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382405632:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572574.821435:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.821640:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360666368 00000100:00100000:1.0:1788572574.821650:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360666368 00010000:00010000:1.0:1788572574.821704:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 248 pid: 5837 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.821746:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1152 00000100:00100000:1.0:1788572574.821758:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2350:x1875453360666368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572574.821778:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572574.821787:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 248 pid: 5837 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.821808:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 248 pid: 5837 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.821854:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f43 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17dd expref: 248 pid: 5837 timeout: 1022 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.821874:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572574.821880:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.821893:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.821902:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572574.821933:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572574.822000:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2350:x1875453360666368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 244us (1095us total) trans 0 rc 0/0 00010000:00010000:0.0:1788572574.822390:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.822400:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.822430:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.822443:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.822451:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572574.823502:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1152 00000004:00080000:3.0:1788572574.832513:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5e:0x0], index 0: 0 00010000:00010000:3.0:1788572574.834271:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb165f82 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834362:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb165f82 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834380:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb165f82 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834442:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834449:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834456:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834462:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572574.834467:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834475:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834490:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572574.834493:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572574.834499:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834505:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb165f74 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572574.834524:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000498def7b reply state 00000000f3031d2f transno 4294971018 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572574.834533:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572574.834595:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f600/0x85ace67feb165f6d lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f2 expref: 66 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834612:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb165f82 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f2 expref: 65 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572574.834620:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572574.834623:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572574.834626:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4600, rc 0) 00000100:00100000:3.0:1788572574.834738:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37303:x1875453360666240:12345-192.168.202.40@tcp:101:dd.0 Request processed in 21831us (22164us total) trans 4294971018 rc 0/0 00000100:00100000:3.0:1788572574.834749:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4936 00010000:00010000:3.0:1788572574.843512:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572574.850058:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.850084:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.850092:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.850099:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54ea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572574.850105:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.850114:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.850144:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627c5160100) 00010000:00010000:0.0:1788572574.850148:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572574.850154:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.850161:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54ea00/0x85ace67feb165f7b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572574.866943:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.867556:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360666496 00000100:00100000:0.0:1788572574.867570:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360666496 00000100:00100000:0.0:1788572574.867649:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1442 00000100:00100000:0.0:1788572574.867662:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:37303:x1875453360666496:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572574.867686:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572574.867716:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f17f9 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572574.867752:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572574.867783:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=922s v=(0 0) 00010000:00010000:0.0:1788572574.867788:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572574.867800:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.867830:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572574.867846:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb54ea00, rc 0) 00000100:00100000:0.0:1788572574.867925:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:37303:x1875453360666496:12345-192.168.202.40@tcp:101:dd.0 Request processed in 266us (998us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572574.867933:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1442 00000100:00100000:3.0:1788572574.896978:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.897477:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360666624 00000100:00100000:0.0:1788572574.897491:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360666624 00002000:00100000:0.0:1788572574.897570:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5e:0x0] (0->1048575) req@ffff9626c5431f80 x1875453360666624/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:235/0 lens 488/0 e 0 to 0 dl 1788572585 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572574.897583:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431f80 with x1875453360666624 ext(0->1048575) 00002000:00010000:0.0:1788572574.897590:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c5431f80 00000100:00100000:0.0:1788572574.897632:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 376 00000100:00100000:0.0:1788572574.897644:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360666624:12345-192.168.202.40@tcp:4:dd.0 00000100:00100000:3.0:1788572574.932217:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5431f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360666624:12345-192.168.202.40@tcp:4:dd.0 Request processed in 34552us (35234us total) trans 4294967675 rc 0/0 00000100:00100000:3.0:1788572574.932252:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 376 00002000:00100000:3.0:1788572574.932274:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5e:0x0] (0->1048575) req@ffff9626c5431f80 x1875453360666624/t4294967675(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:235/0 lens 488/448 e 0 to 0 dl 1788572585 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572574.932286:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5431f80 with x1875453360666624 ext(0->1048575) 00002000:00010000:3.0:1788572574.932605:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c5431f80 00000100:00100000:2.0:1788572574.941392:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572574.942988:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360666752 00000100:00100000:0.0:1788572574.943084:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360666752 00000100:00100000:0.0:1788572574.943413:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4937 00000100:00100000:0.0:1788572574.943429:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37303:x1875453360666752:12345-192.168.202.40@tcp:44:dd.0 00000001:00080000:3.0:1788572574.997900:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971018 is committed 00000001:00080000:3.0:1788572574.997926:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971017 is committed 00000100:00100000:2.0:1788572574.998589:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37303:x1875453360666752:12345-192.168.202.40@tcp:44:dd.0 Request processed in 55153us (57589us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572574.998604:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4937 00000100:00100000:0.0:1788572575.002337:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.002489:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360666880 00000100:00100000:2.0:1788572575.002502:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360666880 00000100:00100000:2.0:1788572575.002596:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1443 00000100:00100000:2.0:1788572575.002611:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360666880:12345-192.168.202.40@tcp:16:dd.0 00000100:00080000:1.0:1788572575.002967:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:3.0:1788572575.037825:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572575.037834:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572575.037881:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 173 count 7 00000040:00080000:3.0:1788572575.037889:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572575.039136:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382405760:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572575.039394:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788acc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572575.048587:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382405760:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572575.061378:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.061458:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.061624:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360667008 00000100:00100000:2.0:1788572575.061634:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360667008 00000100:00100000:2.0:1788572575.061682:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360667136 00000100:00100000:2.0:1788572575.061686:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360667136 00000100:00100000:2.0:1788572575.061702:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4938 00000100:00100000:2.0:1788572575.061711:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37304:x1875453360667008:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572575.061743:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xad:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572575.061848:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37304:x1875453360667008:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 137us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.061856:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4938 00000100:00100000:2.0:1788572575.061877:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4939 00000100:00100000:2.0:1788572575.061884:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37305:x1875453360667136:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572575.061896:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xae:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572575.061930:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+63:37305:x1875453360667136:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 47us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.061937:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4939 00000100:00100000:1.0:1788572575.073050:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.073141:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.073172:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360667392 00000100:00100000:1.0:1788572575.073181:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360667392 00000100:00100000:1.0:1788572575.073235:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360667648 00000100:00100000:1.0:1788572575.073238:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360667648 00000100:00100000:1.0:1788572575.073274:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4940 00000100:00100000:1.0:1788572575.073283:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+64:37305:x1875453360667392:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572575.073305:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572575.073326:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f97 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1800 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073437:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f9e lrc: 3/1,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073453:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f9e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073467:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f9e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572575.073479:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xae:0x0]: returning size 1048576 00010000:00010000:1.0:1788572575.073778:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165f97 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1800 expref: 67 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073798:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f9e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1800 expref: 66 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073805:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:1.0:1788572575.073808:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572575.073811:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3337600, rc 0) 00000100:00100000:1.0:1788572575.073870:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37305:x1875453360667392:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 587us (829us total) trans 4294971019 rc 0/0 00000100:00100000:1.0:1788572575.073878:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4940 00000100:00100000:1.0:1788572575.073899:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4941 00000100:00100000:1.0:1788572575.073906:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37304:x1875453360667648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572575.073914:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572575.073922:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fac lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1807 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073968:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165fb3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073977:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165fb3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.073985:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165fb3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572575.073993:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xad:0x0]: returning size 1048576 00010000:00010000:1.0:1788572575.074074:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fac lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1807 expref: 68 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.074087:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165fb3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1807 expref: 67 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.074095:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:1.0:1788572575.074097:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572575.074100:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e3336400, rc 0) 00000100:00100000:1.0:1788572575.074122:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37304:x1875453360667648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 217us (980us total) trans 4294971020 rc 0/0 00000100:00100000:1.0:1788572575.074128:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4941 00000001:00080000:1.0:1788572575.078399:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967675 is committed 00000100:00100000:1.0:1788572575.078523:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360666880:12345-192.168.202.40@tcp:16:dd.0 Request processed in 75913us (76198us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572575.078534:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1443 00000100:00100000:1.0:1788572575.082695:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.088421:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360667776 00000100:00100000:0.0:1788572575.088437:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360667776 00000100:00100000:0.0:1788572575.088526:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1444 00000100:00100000:0.0:1788572575.088555:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360667776:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00000100:00080000:1.0:1788572575.088593:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1788572575.088604:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572575.088850:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.088889:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.088918:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.088939:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.088964:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360668032 00000100:00100000:2.0:1788572575.088969:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360668032 00000100:00100000:2.0:1788572575.088996:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1445 00000100:00100000:2.0:1788572575.089019:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360668032:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1788572575.089041:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089057:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=1s v=(1 0) 00010000:00010000:2.0:1788572575.089061:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1788572575.089068:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089083:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089121:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089131:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089140:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089153:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6b8600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572575.089158:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089168:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089184:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.089191:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a12cddcc/0x85ace67feb165fc8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572575.089365:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360667904 00000100:00100000:3.0:1788572575.089370:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360667904 00000100:00100000:3.0:1788572575.089407:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360668160 00000100:00100000:3.0:1788572575.089410:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360668160 00000100:00100000:3.0:1788572575.089431:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 377 00000100:00100000:3.0:1788572575.089438:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:2348:x1875453360667904:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572575.089450:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572575.089474:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165f9e lrc: 0/0,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1800 expref: 68 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.089489:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:3.0:1788572575.089494:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572575.089526:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2348:x1875453360667904:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 88us (678us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.089549:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 377 00000100:00100000:3.0:1788572575.089583:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 378 00000100:00100000:1.0:1788572575.089590:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360668416 00000100:00100000:3.0:1788572575.089600:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360668160:12345-192.168.202.40@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1788572575.089605:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360668416 00010000:00010000:3.0:1788572575.089623:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572575.089637:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336400/0x85ace67feb165fb3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1807 expref: 67 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.089651:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:3.0:1788572575.089656:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572575.089688:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1071 00000100:00100000:3.0:1788572575.089689:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:2347:x1875453360668160:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 91us (764us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.089707:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 378 00000100:00100000:1.0:1788572575.089714:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37303:x1875453360668416:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:2.0:1788572575.089855:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360668032:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 848us (961us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.089864:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1445 00010000:00010000:0.0:1788572575.089962:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1788572575.089966:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572575.089972:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.089988:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090042:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090052:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090061:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090069:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54f600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572575.090073:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090081:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090093:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.090102:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0580b65/0x85ace67feb165fc1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572575.090140:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360667776:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 1603us (7892us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572575.090147:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1444 00000100:00100000:1.0:1788572575.090434:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37303:x1875453360668416:12345-192.168.202.40@tcp:35:dd.0 Request processed in 726us (1489us total) trans 4294971021 rc 0/0 00000100:00100000:1.0:1788572575.090444:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1071 00000100:00100000:0.0:1788572575.100522:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.100655:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360668672 00000100:00100000:1.0:1788572575.100667:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360668672 00000100:00100000:1.0:1788572575.100735:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4942 00000100:00100000:1.0:1788572575.100746:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37304:x1875453360668672:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572575.100865:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37304:x1875453360668672:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 121us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572575.100872:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4942 00000100:00100000:1.0:1788572575.105714:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.106184:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360668928 00000100:00100000:2.0:1788572575.106194:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360668928 00000100:00100000:2.0:1788572575.106265:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4943 00000100:00100000:2.0:1788572575.106276:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37305:x1875453360668928:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572575.106398:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37305:x1875453360668928:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 123us (694us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.106405:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4943 00000100:00100000:2.0:1788572575.118697:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.118810:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.128889:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360669056 00000100:00100000:3.0:1788572575.128906:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360669056 00000100:00100000:3.0:1788572575.128978:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360669184 00000100:00100000:3.0:1788572575.128982:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360669184 00000100:00100000:3.0:1788572575.129205:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1446 00000100:00100000:3.0:1788572575.129216:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360669056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572575.129236:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572575.129256:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f180e expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.129281:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572575.129286:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572575.129296:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.129314:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.129329:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4a00, rc 0) 00000100:00100000:3.0:1788572575.129397:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360669056:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 182us (10711us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.129404:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1446 00000100:00100000:3.0:1788572575.129423:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1447 00000100:00100000:3.0:1788572575.129431:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360669184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572575.129439:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572575.129451:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1815 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.129466:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572575.129470:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572575.129475:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 11 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.129488:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 11 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.129499:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4e00, rc 0) 00000100:00100000:3.0:1788572575.129518:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360669184:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 88us (10709us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.129524:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1447 00000100:00100000:0.0:1788572575.144852:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.144889:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.145359:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360669440 00000100:00100000:1.0:1788572575.145369:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360669440 00000100:00100000:1.0:1788572575.145431:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360669696 00000100:00100000:1.0:1788572575.145435:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360669696 00000100:00100000:1.0:1788572575.145463:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4944 00000100:00100000:1.0:1788572575.145473:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37304:x1875453360669440:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572575.145600:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37304:x1875453360669440:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 127us (757us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572575.145608:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4944 00000100:00100000:1.0:1788572575.145626:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4945 00000100:00100000:1.0:1788572575.145633:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37305:x1875453360669696:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572575.145669:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37305:x1875453360669696:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 36us (779us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572575.145675:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4945 00000100:00100000:2.0:1788572575.163686:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.163735:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.163827:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360669824 00000100:00100000:0.0:1788572575.163838:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360669824 00002000:00100000:0.0:1788572575.163893:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5d:0x0] (0->1048575) req@ffff9626c4b31180 x1875453360669824/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:236/0 lens 488/0 e 0 to 0 dl 1788572586 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572575.163907:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b31180 with x1875453360669824 ext(0->1048575) 00002000:00010000:0.0:1788572575.163913:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b31180 00000100:00100000:0.0:1788572575.163941:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360669952 00000100:00100000:0.0:1788572575.163944:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360669952 00002000:00100000:0.0:1788572575.163962:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5d:0x0] (0->1048575) req@ffff9626c4b31880 x1875453360669952/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:236/0 lens 488/0 e 0 to 0 dl 1788572586 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572575.163969:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b31880 with x1875453360669952 ext(0->1048575) 00002000:00010000:0.0:1788572575.163973:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4b31880 00000100:00100000:0.0:1788572575.164037:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 377 00000100:00100000:0.0:1788572575.164047:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360669824:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1788572575.164450:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 378 00000100:00100000:1.0:1788572575.164463:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360669952:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1788572575.186572:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.186751:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670208 00000100:00100000:1.0:1788572575.186773:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670208 00000100:00100000:1.0:1788572575.186864:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4946 00000100:00100000:1.0:1788572575.186877:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37309:x1875453360670208:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:1.0:1788572575.186900:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572575.186924:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165fdd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f181c expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187053:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187069:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187082:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187098:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1788572575.187102:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788572575.187105:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187111:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187493:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187504:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.187512:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572575.187523:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xaf:0x0]: returning size 1048576 00010000:00010000:3.0:1788572575.188301:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188312:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188321:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188327:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572575.188333:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188341:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188355:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572575.188358:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572575.188363:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188370:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337a00/0x85ace67feb165fe4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188387:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188393:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188399:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188405:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e3337600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572575.188408:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188414:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188421:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627ff5abf00) 00010000:00010000:3.0:1788572575.188424:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572575.188427:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188434:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3337600/0x85ace67feb165feb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188485:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e3336800/0x85ace67feb165fdd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 248 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188500:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 247 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.188508:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572575.188510:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572575.188513:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bc400, rc 0) 00000100:00100000:3.0:1788572575.188634:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37309:x1875453360670208:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 1759us (2073us total) trans 4294971022 rc 0/0 00000100:00100000:3.0:1788572575.188644:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4946 00000100:00100000:1.0:1788572575.248841:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.249154:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360669952:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 Request processed in 84693us (85415us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788572575.249166:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 378 00002000:00100000:3.0:1788572575.249179:0:6686:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5d:0x0] (0->1048575) req@ffff9626c4b31880 x1875453360669952/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:236/0 lens 488/440 e 0 to 0 dl 1788572586 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572575.249189:0:6686:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b31880 with x1875453360669952 ext(0->1048575) 00002000:00010000:3.0:1788572575.249197:0:6686:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9626c4b31880 00000100:00100000:2.0:1788572575.249940:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670336 00000100:00100000:2.0:1788572575.249950:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670336 00000100:00100000:2.0:1788572575.250061:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1448 00000100:00100000:2.0:1788572575.250071:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360670336:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572575.250087:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572575.250120:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d88a5eb9/0x85ace67feb166000 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1823 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572575.250180:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382405888:192.168.202.40@tcp:106: 00000100:00100000:2.0:1788572575.250270:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56840 going to sleep for 11 seconds 00000100:00080000:3.0:1788572575.250520:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1788572575.282585:0:10416:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x5e:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:2.0:1788572575.282594:0:10416:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x5e:0x0] updating lvb mtime: 0 -> 1788572574 00002000:00010000:2.0:1788572575.282597:0:10416:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x5e:0x0] updating lvb atime: 0 -> 1788572574 00002000:00010000:2.0:1788572575.282599:0:10416:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x5e:0x0] updating lvb ctime: 0 -> 1788572574 00002000:00010000:2.0:1788572575.282602:0:10416:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x5e:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:2.0:1788572575.282629:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382405888:192.168.202.40@tcp:106: 00010000:00010000:2.0:1788572575.282673:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d88a5eb9/0x85ace67feb166000 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1823 expref: 74 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.282699:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d88a5eb9/0x85ace67feb166000 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1823 expref: 74 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.282719:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b8400, rc 0) 00000100:00100000:2.0:1788572575.282791:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360670336:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 32719us (33959us total) trans 0 rc 301/301 00000100:00100000:2.0:1788572575.282800:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1448 00000100:00100000:1.0:1788572575.296304:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.296488:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670464 00000100:00100000:3.0:1788572575.296499:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670464 00000100:00100000:3.0:1788572575.296586:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4947 00000100:00100000:3.0:1788572575.296596:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+66:37305:x1875453360670464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572575.296616:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1788572575.296626:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360669824:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 Request processed in 132580us (132950us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788572575.296635:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 377 00010000:00010000:3.0:1788572575.296636:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb166007 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f182a expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:2.0:1788572575.296645:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5d:0x0] (0->1048575) req@ffff9626c4b31180 x1875453360669824/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:236/0 lens 488/440 e 0 to 0 dl 1788572586 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572575.296655:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b31180 with x1875453360669824 ext(0->1048575) 00002000:00010000:2.0:1788572575.296660:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b31180 00010000:00010000:3.0:1788572575.296682:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 3/1,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.296695:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.296706:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.296810:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5c00/0x85ace67feb166007 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.296827:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 67 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.296835:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:3.0:1788572575.296838:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572575.296841:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5200, rc 0) 00000100:00100000:3.0:1788572575.296880:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37305:x1875453360670464:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 285us (585us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.296886:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4947 00000100:00100000:2.0:1788572575.304620:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.304905:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670592 00000100:00100000:3.0:1788572575.304916:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670592 00000100:00100000:3.0:1788572575.304982:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1449 00000100:00100000:3.0:1788572575.304992:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:37309:x1875453360670592:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572575.305346:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572575.305370:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1831 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.305408:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.305418:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x5e:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788572575.305422:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572575.305428:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 74 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.305464:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.305477:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 11 pid: 13272 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572575.305493:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382406016:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572575.305587:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e788acc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572575.305701:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.306548:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670720 00000100:00100000:3.0:1788572575.306555:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670720 00000100:00100000:3.0:1788572575.306671:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4948 00000100:00100000:3.0:1788572575.306681:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37305:x1875453360670720:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572575.306709:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xae:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572575.306815:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7874e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37305:x1875453360670720:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 135us (1114us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.306823:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4948 00000100:00100000:0.0:1788572575.316666:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5432300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382406016:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572575.316708:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 3/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 74 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.316730:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5c00, rc 0) 00000100:00100000:0.0:1788572575.316800:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:37309:x1875453360670592:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 11809us (12192us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572575.316810:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1449 00000100:00100000:1.0:1788572575.319686:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.334565:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.334611:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.334705:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670976 00000100:00100000:1.0:1788572575.334718:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670976 00000100:00100000:1.0:1788572575.334786:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1450 00000100:00100000:1.0:1788572575.334795:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360670976:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572575.334867:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.334895:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572575.334899:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572575.334906:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.334921:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.334958:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.334967:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.334975:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.334983:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572575.334988:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.335026:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.335043:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.335052:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000689f751f/0x85ace67feb16601c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572575.335123:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360670976:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 328us (568us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572575.335130:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1450 00000100:00100000:2.0:1788572575.335181:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360670848 00000100:00100000:2.0:1788572575.335186:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360670848 00000100:00100000:2.0:1788572575.335223:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4949 00000100:00100000:2.0:1788572575.335231:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37304:x1875453360670848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1788572575.335242:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360671104 00000100:00100000:1.0:1788572575.335246:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360671104 00010000:00010000:2.0:1788572575.335246:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572575.335264:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb166023 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1838 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.335266:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 12 pid: 13272 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.335293:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.335305:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.335314:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.335413:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8600/0x85ace67feb166023 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 69 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.335430:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 68 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.335437:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:2.0:1788572575.335440:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572575.335443:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b8800, rc 0) 00000100:00100000:2.0:1788572575.335472:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7876300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37304:x1875453360670848:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 243us (15796us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.335479:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4949 00000100:00100000:1.0:1788572575.338121:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1153 00000100:00100000:1.0:1788572575.338146:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2efe050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360671104:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572575.338169:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572575.338198:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 12 pid: 13272 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.338219:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 12 pid: 13272 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.338244:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad478b03/0x85ace67feb165f90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f9 expref: 12 pid: 13272 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.338262:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572575.338268:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572575.338404:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.338416:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.338462:0:5823:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572575.338479:0:5823:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb30e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382406144:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572575.338587:0:5823:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e75b8e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572575.341644:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.341692:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.341767:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360671232 00000100:00100000:3.0:1788572575.341778:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360671232 00000100:00100000:3.0:1788572575.341831:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360671488 00000100:00100000:3.0:1788572575.341835:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360671488 00000100:00100000:3.0:1788572575.341861:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4950 00000100:00100000:3.0:1788572575.341870:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37304:x1875453360671232:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572575.341904:0:5837:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xad:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572575.342046:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37304:x1875453360671232:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 176us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.342053:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4950 00000100:00100000:3.0:1788572575.342072:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4951 00000100:00100000:3.0:1788572575.342079:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37305:x1875453360671488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572575.342453:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.342474:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.342484:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.342520:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 68 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.342552:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 68 pid: 22536 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572575.342566:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c5433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382406272:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572575.342630:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572575.350210:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.350260:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.350299:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.350385:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360671744 00000100:00100000:3.0:1788572575.350395:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360671744 00010000:00010000:3.0:1788572575.350435:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 69 pid: 22536 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572575.350484:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1154 00000100:00100000:3.0:1788572575.350494:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2348:x1875453360671744:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572575.350510:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572575.350517:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 69 pid: 22536 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.350541:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 69 pid: 22536 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.350563:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:3.0:1788572575.350569:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.350578:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.350589:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.350609:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572575.350658:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2348:x1875453360671744:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 166us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.350666:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1154 00000100:00100000:3.0:1788572575.350711:0:5823:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb30e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5823:1875453382406144:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572575.350730:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572575.350748:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2efe050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360671104:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 12615us (16137us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.350754:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1153 00000100:00100000:0.0:1788572575.353644:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360671616 00000100:00100000:0.0:1788572575.353660:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360671616 00002000:00100000:0.0:1788572575.353723:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5e:0x0] (0->32767) req@ffff9627fb30ea00 x1875453360671616/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:236/0 lens 488/0 e 0 to 0 dl 1788572586 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572575.353743:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30ea00 with x1875453360671616 ext(0->32767) 00002000:00010000:0.0:1788572575.353749:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30ea00 00000100:00100000:0.0:1788572575.353794:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 379 00000100:00100000:0.0:1788572575.353803:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2347:x1875453360671616:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:0.0:1788572575.354072:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360671872 00000100:00100000:0.0:1788572575.354076:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360671872 00000100:00100000:0.0:1788572575.354109:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1451 00000100:00100000:0.0:1788572575.354116:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360671872:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572575.354146:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354163:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572575.354167:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572575.354174:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354190:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354216:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354224:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354238:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354245:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572575.354250:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354259:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354274:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354281:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5326c2a/0x85ace67feb166038 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.354312:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16600e lrc: 0/0,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f182a expref: 68 pid: 22536 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572575.354326:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360671872:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 211us (4065us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.354327:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c5433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382406272:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572575.354334:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1451 00010000:00010000:2.0:1788572575.354354:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.354363:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.354369:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354580:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354589:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354595:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354601:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572575.354605:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354611:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354623:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:0.0:1788572575.354626:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572575.354631:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.354637:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166031 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572575.354730:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37305:x1875453360671488:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 12650us (13036us total) trans 4294971023 rc 0/0 00000100:00100000:0.0:1788572575.354739:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4951 00000100:00100000:0.0:1788572575.382437:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2347:x1875453360671616:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 28622us (32225us total) trans 0 rc 32768/32768 00000100:00100000:0.0:1788572575.382452:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 379 00002000:00100000:0.0:1788572575.382465:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x5e:0x0] (0->32767) req@ffff9627fb30ea00 x1875453360671616/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:236/0 lens 488/440 e 0 to 0 dl 1788572586 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572575.382475:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb30ea00 with x1875453360671616 ext(0->32767) 00002000:00010000:0.0:1788572575.382482:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb30ea00 00000100:00100000:3.0:1788572575.391688:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.391786:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.391888:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360672384 00000100:00100000:0.0:1788572575.391899:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360672384 00000100:00100000:0.0:1788572575.391961:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4952 00000100:00100000:0.0:1788572575.391972:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37304:x1875453360672384:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572575.392099:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.392118:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 68 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.392128:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 4/0,1 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.392162:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 68 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572575.392176:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 68 pid: 5838 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572575.392192:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bbb100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382406400:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572575.392263:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572575.392575:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360672128 00000100:00100000:2.0:1788572575.392580:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360672128 00000100:00100000:2.0:1788572575.392619:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1072 00000100:00100000:2.0:1788572575.392628:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37305:x1875453360672128:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572575.392838:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37305:x1875453360672128:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 210us (1158us total) trans 4294971024 rc 0/0 00000100:00100000:2.0:1788572575.392847:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1072 00000100:00100000:3.0:1788572575.401687:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.401815:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bbb100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382406400:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572575.402036:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360672512 00000100:00100000:2.0:1788572575.402045:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360672512 00002000:00100000:2.0:1788572575.402107:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5e:0x0] (32768->1048575) req@ffff9626c5432300 x1875453360672512/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:236/0 lens 488/0 e 0 to 0 dl 1788572586 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572575.402117:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5432300 with x1875453360672512 ext(32768->1048575) 00002000:00010000:2.0:1788572575.402123:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c5432300 00000100:00100000:2.0:1788572575.402159:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 380 00000100:00100000:2.0:1788572575.402169:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360672512:12345-192.168.202.40@tcp:3:md5sum.0 00010000:00010000:0.0:1788572575.402603:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572575.403205:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.409785:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360672640 00000100:00100000:3.0:1788572575.409801:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360672640 00010000:00010000:3.0:1788572575.409845:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 70 pid: 5838 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572575.409898:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1155 00000100:00100000:3.0:1788572575.409908:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2349:x1875453360672640:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572575.409924:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572575.409930:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 70 pid: 5838 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.409948:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 70 pid: 5838 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.409969:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8800/0x85ace67feb16602a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1838 expref: 70 pid: 5838 timeout: 1023 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.409986:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:3.0:1788572575.409991:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.410040:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.410048:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572575.410082:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572575.410145:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:2349:x1875453360672640:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 239us (6940us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.410153:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1155 00010000:00010000:1.0:1788572575.410283:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410293:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410491:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410499:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410510:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410516:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb54fc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572575.410520:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410540:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410550:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:1.0:1788572575.410553:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572575.410558:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 2/0,0 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.410565:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54fc00/0x85ace67feb16603f lrc: 0/0,0 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572575.410708:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37304:x1875453360672384:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 18738us (18921us total) trans 4294971025 rc 0/0 00000100:00100000:1.0:1788572575.410717:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4952 00000040:00080000:3.0:1788572575.410953:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572575.410958:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572575.412913:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 7 00000040:00080000:3.0:1788572575.412930:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1788572575.436843:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572575.437158:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360672896 00000100:00100000:0.0:1788572575.437170:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360672896 00000100:00100000:0.0:1788572575.437225:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1073 00000100:00100000:0.0:1788572575.437235:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec038850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37304:x1875453360672896:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572575.437479:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec038850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37304:x1875453360672896:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 244us (644us total) trans 4294971026 rc 0/0 00000100:00100000:0.0:1788572575.437487:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1073 00000100:00100000:2.0:1788572575.463324:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360672512:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 61144us (61634us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572575.463339:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 380 00002000:00100000:2.0:1788572575.463352:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x5e:0x0] (32768->1048575) req@ffff9626c5432300 x1875453360672512/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:236/0 lens 488/440 e 0 to 0 dl 1788572586 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572575.463363:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c5432300 with x1875453360672512 ext(32768->1048575) 00002000:00010000:2.0:1788572575.463371:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c5432300 00000100:00100000:0.0:1788572575.497605:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.497778:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360673152 00000100:00100000:2.0:1788572575.497793:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360673152 00000100:00100000:2.0:1788572575.497865:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1074 00000100:00100000:2.0:1788572575.497876:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37309:x1875453360673152:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:2.0:1788572575.498769:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37309:x1875453360673152:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 889us (1169us total) trans 4294971027 rc 0/0 00000100:00100000:2.0:1788572575.498779:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1074 00000100:00100000:3.0:1788572575.542991:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572575.543213:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360673280 00000100:00100000:2.0:1788572575.543235:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360673280 00000100:00100000:2.0:1788572575.543317:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4953 00000100:00100000:2.0:1788572575.543329:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37311:x1875453360673280:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572575.543362:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572575.543391:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8c00/0x85ace67feb166046 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f183f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.543471:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.543487:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.543498:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572575.543552:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572575.543556:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572575.543576:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.543614:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b8c00/0x85ace67feb166046 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 250 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.543634:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 249 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572575.543644:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572575.543647:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572575.543650:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b9c00, rc 0) 00000100:00100000:2.0:1788572575.543707:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37311:x1875453360673280:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 379us (728us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572575.543714:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4953 00000100:00100000:0.0:1788572575.550614:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572575.551480:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360673536 00000100:00100000:3.0:1788572575.551493:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360673536 00000100:00100000:3.0:1788572575.551566:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4954 00000100:00100000:3.0:1788572575.551576:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37311:x1875453360673536:12345-192.168.202.40@tcp:58:lfs.0 00000040:00080000:3.0:1788572575.552470:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].173, 136 off31512 00000100:00100000:3.0:1788572575.552646:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37311:x1875453360673536:12345-192.168.202.40@tcp:58:lfs.0 Request processed in 1068us (2041us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572575.552656:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4954 00000100:00100000:3.0:1788572575.854449:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572575.854539:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360673664 00000100:00100000:1.0:1788572575.854550:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360673664 00000100:00100000:1.0:1788572575.854613:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 412 00000100:00100000:1.0:1788572575.854632:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:2349:x1875453360673664:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572575.854669:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572575.854691:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166054 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1846 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.854749:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 3/1,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.854765:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.854777:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.854792:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.854799:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.854806:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855180:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855189:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855197:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855202:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572575.855206:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855219:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855230:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572575.855233:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572575.855237:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855243:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16605b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855255:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855260:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855266:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855271:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572575.855274:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855279:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855285:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x8799b2a3 (ffff9627ff5abe00) 00010000:00010000:1.0:1788572575.855287:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572575.855289:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855298:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166062 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xad:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855329:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166054 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1846 expref: 250 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855338:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffff9627c2abc300) 00010000:00010000:1.0:1788572575.855340:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572575.855344:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166054 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1846 expref: 250 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572575.855352:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647800, rc -2) 00000100:00100000:1.0:1788572575.855432:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:2349:x1875453360673664:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 809us (992us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572575.855440:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 412 00010000:00010000:3.0:1788572575.866270:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572576.370417:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.370455:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.370474:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.370513:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.370536:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.370558:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.370586:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360673920 00000100:00100000:2.0:1788572576.370597:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360673920 00000100:00100000:2.0:1788572576.370646:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674432 00000100:00100000:2.0:1788572576.370650:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674432 00000100:00100000:2.0:1788572576.370667:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674048 00000100:00100000:2.0:1788572576.370669:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674048 00000100:00100000:2.0:1788572576.370690:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674304 00000100:00100000:2.0:1788572576.370693:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674304 00000100:00100000:2.0:1788572576.370708:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 627 00000100:00100000:2.0:1788572576.370717:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360673920:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572576.370783:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360673920:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 66us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.370792:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 627 00000100:00100000:2.0:1788572576.370812:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 628 00000100:00100000:2.0:1788572576.370819:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360674432:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572576.370847:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360674432:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 29us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.370853:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 628 00000100:00100000:2.0:1788572576.370864:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 629 00000100:00100000:2.0:1788572576.370870:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2350:x1875453360674048:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572576.370893:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2350:x1875453360674048:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 24us (357us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.370900:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 629 00000100:00100000:2.0:1788572576.370914:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 630 00000100:00100000:2.0:1788572576.370921:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360674304:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572576.370947:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360674304:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 27us (388us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.370954:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 630 00000100:00100000:2.0:1788572576.370971:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674176 00000100:00100000:2.0:1788572576.370975:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674176 00000100:00100000:2.0:1788572576.371038:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 657 00000100:00100000:2.0:1788572576.371045:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360674176:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572576.371090:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:2350:x1875453360674176:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 45us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.371097:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 657 00000100:00100000:2.0:1788572576.371120:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360673792 00000100:00100000:2.0:1788572576.371123:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360673792 00000100:00100000:2.0:1788572576.371148:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 691 00000100:00100000:2.0:1788572576.371155:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2350:x1875453360673792:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572576.371180:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2350:x1875453360673792:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 25us (666us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.371186:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 691 00000100:00080000:2.0:1788572576.371223:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788572576.371228:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000040:00080000:0.0:1788572576.442351:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572576.442361:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572576.442403:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 174 count 8 00000040:00080000:0.0:1788572576.442412:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1788572576.442686:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bb8380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382406528:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572576.442778:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572576.461779:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.461851:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674560 00000100:00100000:2.0:1788572576.461861:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674560 00000100:00100000:2.0:1788572576.461928:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4955 00000100:00100000:2.0:1788572576.461939:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:37321:x1875453360674560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572576.461972:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xaf:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572576.462101:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:37321:x1875453360674560:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 163us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.462109:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4955 00000100:00100000:0.0:1788572576.466409:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bb8380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382406528:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572576.483423:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572576.491040:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674816 00000100:00100000:0.0:1788572576.491059:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674816 00000100:00100000:0.0:1788572576.491130:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4956 00000100:00100000:0.0:1788572576.491141:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37321:x1875453360674816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572576.491162:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572576.491185:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166069 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f184d expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.491298:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166070 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.491313:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166070 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.491327:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166070 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572576.491338:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xaf:0x0]: returning size 1048576 00010000:00010000:0.0:1788572576.491477:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166069 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f184d expref: 73 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.491511:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166070 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f184d expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.491520:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572576.491524:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572576.491527:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0200, rc 0) 00000100:00100000:0.0:1788572576.491595:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360674816:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 455us (8184us total) trans 4294971028 rc 0/0 00000100:00100000:0.0:1788572576.491602:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4956 00000100:00100000:2.0:1788572576.498257:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572576.500369:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360674944 00000100:00100000:1.0:1788572576.500386:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360674944 00000100:00100000:1.0:1788572576.500452:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1452 00000100:00100000:1.0:1788572576.500463:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360674944:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572576.500519:0:13272:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500536:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=1s v=(1 0) 00010000:00010000:1.0:1788572576.500541:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 1) 00010000:00010000:1.0:1788572576.500548:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500570:0:13272:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500604:0:13272:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500612:0:13272:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500620:0:13272:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500627:0:13272:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572576.500631:0:13272:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500640:0:13272:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500656:0:13272:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.500665:0:13272:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077621369/0x85ace67feb16607e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572576.500766:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360674944:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 304us (2530us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572576.500774:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1452 00000100:00100000:3.0:1788572576.519444:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572576.519549:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.519660:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360675200 00000100:00100000:2.0:1788572576.519671:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360675200 00000100:00100000:2.0:1788572576.519729:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4957 00000100:00100000:2.0:1788572576.519739:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360675200:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572576.519876:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c13bdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360675200:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 137us (440us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.519883:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4957 00000100:00100000:1.0:1788572576.526491:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360675328 00000100:00100000:1.0:1788572576.526508:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360675328 00000100:00100000:3.0:1788572576.529963:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.530243:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360675456 00000100:00100000:2.0:1788572576.530254:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360675456 00000100:00100000:2.0:1788572576.530313:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1453 00000100:00100000:2.0:1788572576.530324:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360675456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572576.530343:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572576.530362:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1854 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.530390:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 1) 00010000:00010000:2.0:1788572576.530395:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 1) 00010000:00010000:2.0:1788572576.530404:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.530421:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 12 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.530436:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6b8000, rc 0) 00000100:00100000:2.0:1788572576.530510:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360675456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 187us (558us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.530518:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1453 00000100:00100000:3.0:1788572576.533661:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.533788:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360675712 00000100:00100000:2.0:1788572576.533798:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360675712 00000100:00100000:2.0:1788572576.533849:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4958 00000100:00100000:2.0:1788572576.533860:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37321:x1875453360675712:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572576.533975:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37321:x1875453360675712:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 116us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.533986:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4958 00000100:00100000:0.0:1788572576.534760:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 379 00000100:00100000:0.0:1788572576.534771:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2347:x1875453360675328:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572576.534786:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572576.534809:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166070 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f184d expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.534826:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572576.534831:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572576.534870:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:2347:x1875453360675328:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 100us (15320us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572576.534877:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 379 00000100:00100000:3.0:1788572576.555107:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.555269:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360675840 00000100:00100000:2.0:1788572576.555280:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360675840 00002000:00100000:2.0:1788572576.555331:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5e:0x0] (0->1048575) req@ffff9626c4b30e00 x1875453360675840/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:237/0 lens 488/0 e 0 to 0 dl 1788572587 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572576.555344:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b30e00 with x1875453360675840 ext(0->1048575) 00002000:00010000:2.0:1788572576.555350:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b30e00 00000100:00100000:2.0:1788572576.555378:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 381 00000100:00100000:2.0:1788572576.555388:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360675840:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1788572576.606912:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360675840:12345-192.168.202.40@tcp:3:lhsmtool_posix.0 Request processed in 51513us (51804us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1788572576.606928:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 381 00002000:00100000:1.0:1788572576.606943:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x5e:0x0] (0->1048575) req@ffff9626c4b30e00 x1875453360675840/t0(0) o3->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:237/0 lens 488/440 e 0 to 0 dl 1788572587 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572576.607342:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c4b30e00 with x1875453360675840 ext(0->1048575) 00002000:00010000:1.0:1788572576.607352:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c4b30e00 00000100:00100000:1.0:1788572576.630875:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572576.631055:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360675968 00000100:00100000:0.0:1788572576.631069:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360675968 00000100:00100000:0.0:1788572576.631135:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4959 00000100:00100000:0.0:1788572576.631147:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:37321:x1875453360675968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572576.631166:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572576.631188:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16608c lrc: 2/0,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f185b expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.631235:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.631254:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.631268:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.631386:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16608c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 73 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.631407:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.631419:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572576.631423:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572576.631427:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1c00, rc 0) 00000100:00100000:0.0:1788572576.631505:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360675968:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 360us (639us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572576.631513:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4959 00000100:00100000:2.0:1788572576.646032:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.646177:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360676096 00000100:00100000:2.0:1788572576.646197:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360676096 00000100:00100000:2.0:1788572576.646255:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4960 00000100:00100000:2.0:1788572576.646266:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360676096:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572576.646303:0:22536:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xaf:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572576.646417:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360676096:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 153us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.646425:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4960 00000100:00100000:0.0:1788572576.651516:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572576.651551:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360676224 00000100:00100000:0.0:1788572576.651562:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360676224 00000100:00100000:0.0:1788572576.651625:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1454 00000100:00100000:0.0:1788572576.651635:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb33e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360676224:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572576.651677:0:6681:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651694:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 1) 00010000:00010000:0.0:1788572576.651698:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 1) 00010000:00010000:0.0:1788572576.651704:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651719:0:6681:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651749:0:6681:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651757:0:6681:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651767:0:6681:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651774:0:6681:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0a00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572576.651778:0:6681:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651794:0:6681:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651809:0:6681:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.651817:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4462e30/0x85ace67feb16609a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572576.651882:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb33e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360676224:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 249us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572576.651890:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1454 00000100:00100000:0.0:1788572576.655130:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572576.655174:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360676480 00000100:00100000:0.0:1788572576.655184:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360676480 00000100:00100000:0.0:1788572576.655244:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4961 00000100:00100000:0.0:1788572576.655254:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360676480:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572576.655349:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.655371:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.655381:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.655424:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.655439:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 72 pid: 5837 timeout: 1024 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572576.655455:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bb8380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382406656:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572576.655562:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32240 going to sleep for 11 seconds 00000100:00100000:0.0:1788572576.666443:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572576.666534:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bb8380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382406656:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572576.666572:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572576.666615:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360676608 00000100:00100000:0.0:1788572576.666621:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360676608 00010000:00010000:0.0:1788572576.666656:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 73 pid: 5837 timeout: 1024 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572576.666688:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1156 00000100:00100000:0.0:1788572576.666699:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:2349:x1875453360676608:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572576.666714:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572576.666721:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 73 pid: 5837 timeout: 1024 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666734:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 73 pid: 5837 timeout: 1024 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666761:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166093 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f185b expref: 73 pid: 5837 timeout: 1024 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666775:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572576.666780:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666790:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666799:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666819:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572576.666863:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:2349:x1875453360676608:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 166us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572576.666871:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1156 00010000:00010000:0.0:1788572576.666896:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.666903:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667134:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667143:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667150:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667157:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572576.667162:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667170:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667180:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572576.667182:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572576.667187:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572576.667194:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb1660a1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572576.667308:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360676480:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 12055us (12193us total) trans 4294971029 rc 0/0 00000100:00100000:0.0:1788572576.667317:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4961 00000040:00080000:0.0:1788572576.667417:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572576.667422:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572576.667908:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 8 00000040:00080000:0.0:1788572576.667916:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1788572576.671933:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.672326:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360676864 00000100:00100000:2.0:1788572576.672336:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360676864 00000100:00100000:2.0:1788572576.672395:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1075 00000100:00100000:2.0:1788572576.672406:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37321:x1875453360676864:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572576.672722:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37321:x1875453360676864:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 317us (798us total) trans 4294971030 rc 0/0 00000100:00100000:2.0:1788572576.672734:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1075 00000040:00080000:3.0:1788572576.808485:0:23798:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572576.808495:0:23798:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572576.808696:0:23798:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 174 count 8 00000040:00080000:3.0:1788572576.808704:0:23798:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1788572576.895156:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572576.946721:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572576.946879:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360676992 00000100:00100000:1.0:1788572576.946893:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360676992 00000100:00100000:1.0:1788572576.946971:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4962 00000100:00100000:1.0:1788572576.946982:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37328:x1875453360676992:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:1.0:1788572576.947040:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572576.947068:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660a8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1862 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.947150:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1660af lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.947170:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1660af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.947186:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1660af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572576.947237:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:1.0:1788572576.947242:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572576.947263:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1660af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.947296:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660a8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1862 expref: 75 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.947313:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1660af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1862 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.947323:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572576.947326:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572576.947329:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646000, rc 0) 00000100:00100000:1.0:1788572576.947402:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37328:x1875453360676992:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 421us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572576.947409:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4962 00000100:00100000:0.0:1788572576.969737:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.969818:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360677248 00000100:00100000:2.0:1788572576.969830:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360677248 00000100:00100000:2.0:1788572576.969912:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4963 00000100:00100000:2.0:1788572576.969923:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37329:x1875453360677248:12345-192.168.202.40@tcp:36:rm.0 00010000:00010000:2.0:1788572576.970041:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572576.970073:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb165c3a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1655 expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.970094:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:2.0:1788572576.970105:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1660af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1862 expref: 73 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.970116:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572576.970120:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572576.970144:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.970160:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 248 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.970169:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.970207:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 248 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.970228:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 248 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572576.970244:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382406784:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572576.970335:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1e51a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572576.979414:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382406784:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572576.979453:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572576.984410:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572576.984507:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360677376 00000100:00100000:3.0:1788572576.984520:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360677376 00010000:00010000:3.0:1788572576.984566:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 249 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572576.984613:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1157 00000100:00100000:3.0:1788572576.984623:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:2349:x1875453360677376:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572576.984644:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572576.984653:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 249 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572576.984672:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 249 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572576.984697:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6b9c00/0x85ace67feb16604d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f183f expref: 249 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572576.984717:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572576.984722:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572576.984732:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572576.984741:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572576.984782:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572576.984837:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2349:x1875453360677376:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 214us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572576.984845:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1157 00010000:00010000:1.0:1788572576.984965:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.984975:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985000:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985042:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985050:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985404:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985416:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 247 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985425:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985456:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 247 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.985482:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 247 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572576.985497:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ea300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382406912:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572576.985558:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d1513f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572576.997602:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572576.997881:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360677504 00000100:00100000:2.0:1788572576.997892:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360677504 00010000:00010000:2.0:1788572576.997947:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 248 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572576.997993:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1158 00000100:00100000:2.0:1788572576.998125:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2350:x1875453360677504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572576.998143:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572576.998151:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 248 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.998164:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 248 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.998187:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:2.0:1788572576.998193:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.998203:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.998363:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572576.998388:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572576.998449:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2350:x1875453360677504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 446us (856us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572576.998457:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1158 00010000:00010000:1.0:1788572576.999593:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9c00/0x85ace67feb165caa lrc: 0/0,0 mode: --/PR res: [0x200000402:0xab:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1678 expref: 247 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572576.999610:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ea300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382406912:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572576.999632:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.999642:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572576.999648:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572577.001662:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].80, 64 off14704 00000004:00080000:1.0:1788572577.001681:0:22536:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 925070676194ns, next at 926070671820ns, rc = 0 00010000:00010000:1.0:1788572577.001793:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 3/0,1 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.001813:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.001828:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.001839:0:22536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.001846:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.001853:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002025:0:22536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788572577.002053:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002061:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002067:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002073:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572577.002078:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002087:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002101:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:1.0:1788572577.002104:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572577.002109:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002116:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660c4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xab:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002137:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002144:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002156:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002162:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572577.002165:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002171:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002179:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572577.002182:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572577.002185:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.002190:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1660b6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572577.002201:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006a32c14a reply state 000000005fa75395 transno 4294971031 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.002280:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572577.002375:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37329:x1875453360677248:12345-192.168.202.40@tcp:36:rm.0 Request processed in 32448us (32643us total) trans 4294971031 rc 0/0 00000100:00100000:1.0:1788572577.002386:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4963 00010000:00010000:2.0:1788572577.007385:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.007405:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.007414:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.007422:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.007427:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.007436:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.007455:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627ff5abe00) 00010000:00010000:2.0:1788572577.007472:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572577.007478:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.007485:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1660bd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.012078:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572577.012091:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.012102:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572577.012106:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.012122:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.012140:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xab:0x0].0x0 (ffff9626c4b10500) 00010000:00010000:3.0:1788572577.012143:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572577.012148:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 1/0,0 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.012155:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1660cb lrc: 0/0,0 mode: --/PW res: [0x200000402:0xab:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.012193:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572577.012195:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572577.012197:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572577.026335:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.026927:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360677760 00000100:00100000:1.0:1788572577.026950:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360677760 00000100:00100000:1.0:1788572577.027046:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4964 00000100:00100000:1.0:1788572577.027057:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37330:x1875453360677760:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:1.0:1788572577.027088:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572577.027110:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660d2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1869 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.027412:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.027429:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.027441:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.027456:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.027474:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.027482:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572577.028743:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x5e:0x0], index 1: 0 00010000:00010000:1.0:1788572577.028925:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.028952:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.028962:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029618:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029630:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029638:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029644:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572577.029648:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029656:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029669:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572577.029672:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572577.029677:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.029683:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660d9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572577.029697:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000014b2dd4d reply state 0000000052d24d23 transno 4294971032 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.029707:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788572577.033573:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb1660d2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 76 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.033601:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 75 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.033610:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab400) 00010000:00010000:0.0:1788572577.033613:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572577.033616:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647400, rc 0) 00000100:00100000:0.0:1788572577.035135:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:37330:x1875453360677760:12345-192.168.202.40@tcp:101:dd.0 Request processed in 8068us (8799us total) trans 4294971032 rc 0/0 00000100:00100000:0.0:1788572577.035151:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4964 00010000:00010000:0.0:1788572577.052156:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.052176:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.052185:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.052193:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.052199:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.052220:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.052241:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627ff5ab700) 00010000:00010000:0.0:1788572577.052244:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.052250:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.052258:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1660e0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.053623:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572577.053693:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360678016 00000100:00100000:2.0:1788572577.053704:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360678016 00000100:00100000:2.0:1788572577.053769:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1076 00000100:00100000:2.0:1788572577.053781:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37330:x1875453360678016:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:2.0:1788572577.054370:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:37330:x1875453360678016:12345-192.168.202.40@tcp:35:dd.0 Request processed in 587us (754us total) trans 4294971033 rc 0/0 00000100:00100000:2.0:1788572577.054378:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1076 00000100:00100000:0.0:1788572577.152514:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.152631:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360678272 00000100:00100000:1.0:1788572577.152641:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360678272 00000100:00100000:1.0:1788572577.152742:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4965 00000100:00100000:1.0:1788572577.152754:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37333:x1875453360678272:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572577.152775:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572577.152798:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660f5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1870 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.152882:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.152900:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.152913:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.152946:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.152955:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.152962:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788572577.153310:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788572577.153615:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153625:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153634:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572577.153647:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb0:0x0]: returning size 0 00010000:00010000:1.0:1788572577.153751:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153759:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153765:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153770:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572577.153775:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153783:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153796:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572577.153800:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572577.153804:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153811:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1660fc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153821:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153827:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153838:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153843:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572577.153846:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153852:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153859:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572577.153861:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572577.153864:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153869:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166103 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153911:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1660f5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 250 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153925:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 249 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.153933:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab400) 00010000:00010000:1.0:1788572577.153935:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572577.154433:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646800, rc 0) 00000100:00100000:1.0:1788572577.154555:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37333:x1875453360678272:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1802us (2049us total) trans 4294971034 rc 0/0 00000100:00100000:1.0:1788572577.154567:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4965 00000100:00100000:2.0:1788572577.162709:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.162862:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360678528 00000100:00100000:3.0:1788572577.162873:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360678528 00000100:00100000:3.0:1788572577.163051:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4966 00000100:00100000:3.0:1788572577.163063:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37333:x1875453360678528:12345-192.168.202.40@tcp:55:lfs.0 00010000:00010000:3.0:1788572577.163133:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.163151:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 75 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.163161:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 249 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.163170:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 5/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.165180:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 249 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.165217:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 249 pid: 5839 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.165236:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382407040:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572577.165377:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 75 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.165388:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 75 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.165398:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b30000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382407168:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572577.165421:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572577.168898:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382407040:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572577.168917:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b30000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382407168:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572577.169343:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.171951:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.172160:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360678656 00000100:00100000:0.0:1788572577.172169:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360678656 00010000:00010000:0.0:1788572577.172205:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 250 pid: 5839 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.172242:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1159 00000100:00100000:0.0:1788572577.172252:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c2eff050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:2350:x1875453360678656:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572577.172267:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572577.172274:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 250 pid: 5839 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.172292:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 250 pid: 5839 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.172318:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb16610a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1870 expref: 250 pid: 5839 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.172334:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab400) 00010000:00010000:0.0:1788572577.172341:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.172350:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.172356:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572577.172414:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c2eff050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:2350:x1875453360678656:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 164us (483us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.172421:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1159 00000100:00100000:2.0:1788572577.172829:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.172885:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360678784 00000100:00100000:3.0:1788572577.172890:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360678784 00010000:00010000:3.0:1788572577.172915:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 76 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.172955:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1160 00000100:00100000:3.0:1788572577.172963:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:2350:x1875453360678784:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.172972:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.172977:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 76 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.172987:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 76 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173027:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1660e7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1869 expref: 76 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173039:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab400) 00010000:00010000:3.0:1788572577.173043:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173051:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173058:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173082:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572577.173105:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2350:x1875453360678784:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 143us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.173112:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1160 00010000:00010000:3.0:1788572577.173136:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173148:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173366:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173374:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173381:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173386:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.173391:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173398:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.173408:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab400) 00010000:00010000:3.0:1788572577.173411:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572577.173415:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.175050:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166118 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.175139:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37333:x1875453360678528:12345-192.168.202.40@tcp:55:lfs.0 Request processed in 12076us (12440us total) trans 4294971035 rc 0/0 00000100:00100000:3.0:1788572577.175148:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4966 00000100:00100000:2.0:1788572577.179244:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.179496:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679040 00000100:00100000:0.0:1788572577.179510:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679040 00000100:00100000:0.0:1788572577.179575:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1077 00000100:00100000:0.0:1788572577.179586:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37333:x1875453360679040:12345-192.168.202.40@tcp:35:lfs.0 00000100:00100000:0.0:1788572577.179855:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:37333:x1875453360679040:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 270us (621us total) trans 4294971036 rc 0/0 00000100:00100000:0.0:1788572577.179863:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1077 00000100:00100000:2.0:1788572577.200225:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.200300:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679168 00000100:00100000:0.0:1788572577.200311:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679168 00000100:00100000:0.0:1788572577.200375:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4967 00000100:00100000:0.0:1788572577.200385:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+250:37334:x1875453360679168:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572577.200409:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572577.200431:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16611f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1877 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200516:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200531:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200542:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200558:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200566:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200572:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200669:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200677:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200685:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572577.200721:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb0:0x0]: returning size 0 00000004:00200000:0.0:1788572577.200726:0:5838:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572577.200746:0:5838:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200758:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200766:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200773:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200782:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200789:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.200794:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200800:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200812:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572577.200815:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.200819:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200825:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166126 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200837:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200843:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200849:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200854:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.200857:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200867:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200874:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627c5160c00) 00010000:00010000:0.0:1788572577.200877:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.200880:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200885:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16612d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200915:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16611f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 252 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200928:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 251 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.200935:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627c5160300) 00010000:00010000:0.0:1788572577.200947:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572577.200950:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0e00, rc 0) 00000100:00100000:0.0:1788572577.201150:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:37334:x1875453360679168:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 766us (935us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.201158:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4967 00000100:00100000:1.0:1788572577.210405:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572577.210855:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679296 00000100:00100000:2.0:1788572577.210864:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679296 00000100:00100000:2.0:1788572577.210939:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1455 00000100:00100000:2.0:1788572577.210950:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360679296:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572577.210967:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572577.210988:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f187e expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572577.211197:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1788572577.211229:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=925s v=(0 0) 00010000:00010000:2.0:1788572577.211235:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572577.211245:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 86 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.211266:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 86 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.211280:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aea00, rc 0) 00000100:00100000:2.0:1788572577.211347:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2349:x1875453360679296:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 399us (953us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572577.211355:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1455 00000100:00100000:2.0:1788572577.212517:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.212992:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679424 00000100:00100000:3.0:1788572577.213021:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679424 00000100:00100000:3.0:1788572577.213077:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 413 00000100:00100000:3.0:1788572577.213086:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:2347:x1875453360679424:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572577.213120:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572577.213138:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166142 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1885 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213198:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213213:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213223:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213239:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213246:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213253:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213307:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213315:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213327:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213333:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.213337:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213345:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213356:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.213359:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.213364:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213370:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166149 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213382:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213388:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213393:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213398:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.213401:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213412:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213419:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xb3dc5f41 (ffff9626c4b10500) 00010000:00010000:3.0:1788572577.213421:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.213424:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213430:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166150 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213475:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166142 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1885 expref: 252 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213485:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.213487:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572577.213491:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166142 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1885 expref: 252 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.213500:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5400, rc -2) 00000100:00100000:3.0:1788572577.213573:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:2347:x1875453360679424:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 487us (1060us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572577.213581:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 413 00000100:00100000:2.0:1788572577.220201:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.221066:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679552 00000100:00100000:0.0:1788572577.221081:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679552 00000100:00100000:0.0:1788572577.221158:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4968 00000100:00100000:0.0:1788572577.221169:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+251:37334:x1875453360679552:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572577.221195:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572577.221253:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166157 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18a1 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.221337:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.221350:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.221359:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572577.221390:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572577.221393:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572577.221406:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.221426:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166157 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 253 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.221440:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 252 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.221446:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572577.221448:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572577.221463:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0800, rc 0) 00000100:00100000:0.0:1788572577.222283:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7558e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:37334:x1875453360679552:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1112us (2088us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.222298:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4968 00000100:00100000:1.0:1788572577.230581:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572577.230679:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679808 00000100:00100000:2.0:1788572577.230689:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679808 00000100:00100000:2.0:1788572577.230805:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4969 00000100:00100000:2.0:1788572577.230816:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+252:37334:x1875453360679808:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572577.230838:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572577.230871:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb166165 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18a8 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.230999:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16616c lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.231050:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16616c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.231061:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16616c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572577.231080:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb0:0x0]: returning size 0 00010000:00010000:2.0:1788572577.231221:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb166165 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a8 expref: 255 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.231241:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16616c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a8 expref: 254 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.231251:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627c5160300) 00010000:00010000:2.0:1788572577.231254:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1788572577.231258:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af400, rc 0) 00000100:00100000:2.0:1788572577.231317:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:37334:x1875453360679808:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 503us (747us total) trans 4294971037 rc 0/0 00000100:00100000:2.0:1788572577.231326:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4969 00000100:00100000:1.0:1788572577.233988:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.234150:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360679936 00000100:00100000:0.0:1788572577.234163:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360679936 00000100:00100000:0.0:1788572577.234232:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1456 00000100:00100000:0.0:1788572577.234244:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2349:x1875453360679936:12345-192.168.202.40@tcp:1:lfs.0 00010000:00010000:0.0:1788572577.234290:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.234312:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 86 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.234320:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572577.234325:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572577.234330:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 4/0,1 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.234369:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 86 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.234385:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 86 pid: 13272 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.234402:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff5ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382407296:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572577.234514:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572577.238611:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.238709:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360680064 00000100:00100000:3.0:1788572577.238721:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360680064 00010000:00010000:3.0:1788572577.238771:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 87 pid: 13272 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.238825:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1161 00000100:00100000:3.0:1788572577.238835:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+87:2350:x1875453360680064:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.238851:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.238881:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 87 pid: 13272 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.238896:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 87 pid: 13272 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.238929:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572577.238934:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572577.238950:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.238962:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.238984:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572577.239064:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+87:2350:x1875453360680064:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 230us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.239072:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1161 00010000:00010000:2.0:1788572577.240668:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb16613b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f187e expref: 86 pid: 13272 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.240697:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff5ef100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382407296:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572577.240726:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240741:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240749:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240791:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240801:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240810:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240818:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572577.240823:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240833:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240850:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.240859:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f30003d0/0x85ace67feb16617a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.241747:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360679936:12345-192.168.202.40@tcp:1:lfs.0 Request processed in 7496us (7764us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.241761:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1456 00000100:00100000:0.0:1788572577.245476:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.245591:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360680320 00000100:00100000:1.0:1788572577.245603:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360680320 00000100:00100000:1.0:1788572577.245785:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1078 00000100:00100000:1.0:1788572577.245798:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13f050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+254:37334:x1875453360680320:12345-192.168.202.40@tcp:35:lfs.0 00000004:00010000:1.0:1788572577.245898:0:5841:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0xb0:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16616c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a8 expref: 254 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572577.246381:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971038 00000001:00080000:1.0:1788572577.249885:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971039 00010000:00010000:1.0:1788572577.250757:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.250781:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 257 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.250790:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.250834:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 257 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.250853:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 257 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.250877:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5841:1875453382407424:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572577.251000:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13ebd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572577.255358:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.255508:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360680448 00000100:00100000:3.0:1788572577.255519:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360680448 00010000:00010000:3.0:1788572577.255568:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 258 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.255619:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1162 00000100:00100000:3.0:1788572577.255630:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+258:2350:x1875453360680448:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.255646:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.255653:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 258 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.255668:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 258 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.255691:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627c5160300) 00010000:00010000:3.0:1788572577.255696:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.255707:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.255714:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.255735:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572577.255828:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+258:2350:x1875453360680448:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 199us (480us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.255837:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1162 00010000:00010000:1.0:1788572577.255945:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166134 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1877 expref: 257 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.255964:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5841:1875453382407424:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572577.255985:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.255994:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.255999:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572577.256168:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971040 00000001:00080000:1.0:1788572577.256372:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971041 00000001:00080000:1.0:1788572577.256430:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971042 00000100:00100000:1.0:1788572577.256522:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5841:1875453382407552:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1788572577.256571:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572577.256595:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ffe82f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572577.256832:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382407552 00000100:00100000:3.0:1788572577.256838:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382407552 00000100:00100000:3.0:1788572577.256899:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1788572577.256907:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5841:x1875453382407552:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1788572577.256993:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5841:x1875453382407552:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 87us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.257000:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00080000:1.0:1788572577.257521:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572577.257552:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5841:1875453382407552:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:1.0:1788572577.257609:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971043 00010000:00010000:1.0:1788572577.257642:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.257656:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.257664:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.257670:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572577.257675:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.257683:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.257697:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627c5160300) 00010000:00010000:1.0:1788572577.257700:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1788572577.257705:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.257711:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166181 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572577.257948:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0xd1:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1788572577.258157:0:5841:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].77, 64 off14512 00000004:00080000:1.0:1788572577.258171:0:5841:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 925327166130ns, next at 926327163565ns, rc = 0 00000001:00080000:1.0:1788572577.258178:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971044 00010000:00010000:1.0:1788572577.258378:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627c5160300) 00010000:00010000:1.0:1788572577.258382:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1788572577.258389:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16616c lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a8 expref: 261 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.258500:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13f050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:37334:x1875453360680320:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 12704us (13045us total) trans 4294971045 rc 0/0 00000100:00100000:1.0:1788572577.258510:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1078 00000020:00080000:1.0:1788572577.258514:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1788572577.322308:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572577.322637:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360680576 00000100:00100000:2.0:1788572577.322660:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360680576 00000100:00100000:2.0:1788572577.322726:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4970 00000100:00100000:2.0:1788572577.322738:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37340:x1875453360680576:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:2.0:1788572577.322765:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572577.323034:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166188 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18af expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.323116:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16618f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.323132:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16618f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.323143:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16618f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572577.323251:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572577.323255:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572577.323278:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16618f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.323307:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166188 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18af expref: 77 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.323324:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16618f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18af expref: 76 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.323332:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572577.323396:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572577.323402:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0c00, rc 0) 00000100:00100000:0.0:1788572577.323483:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:37340:x1875453360680576:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 746us (1187us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.323494:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4970 00000100:00100000:3.0:1788572577.342065:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.342155:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360680832 00000100:00100000:1.0:1788572577.342220:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360680832 00000100:00100000:1.0:1788572577.342284:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4971 00000100:00100000:1.0:1788572577.342292:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:37341:x1875453360680832:12345-192.168.202.40@tcp:36:rm.0 00010000:00010000:1.0:1788572577.342395:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572577.342424:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb165d0c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16a2 expref: 76 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.342443:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:1.0:1788572577.342467:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16618f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18af expref: 75 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.342477:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572577.342481:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572577.342506:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.342529:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 259 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.342538:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.342786:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 259 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.342802:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 259 pid: 5837 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.342818:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382407680:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572577.343264:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ffe82f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572577.345806:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572577.345911:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360680960 00000100:00100000:2.0:1788572577.345923:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360680960 00010000:00010000:2.0:1788572577.345976:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 260 pid: 5837 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.346051:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1163 00000100:00100000:2.0:1788572577.346061:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:2349:x1875453360680960:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572577.346075:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572577.346083:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 260 pid: 5837 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.346098:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 260 pid: 5837 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.346120:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572577.346125:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.346137:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.346145:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.346176:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572577.346238:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:2349:x1875453360680960:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 177us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572577.346251:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1163 00010000:00010000:3.0:1788572577.349469:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16615e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18a1 expref: 259 pid: 5837 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.349502:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12e8000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382407680:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572577.349528:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349539:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349544:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349674:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349691:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349703:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349818:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349831:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 258 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349840:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349878:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 258 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.349895:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 258 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.349910:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382407808:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572577.350278:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572577.353790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.354500:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360681088 00000100:00100000:0.0:1788572577.354511:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360681088 00010000:00010000:0.0:1788572577.354554:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 259 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.354610:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1164 00000100:00100000:0.0:1788572577.354620:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+259:2350:x1875453360681088:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572577.354635:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572577.354643:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 259 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.354656:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 259 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.354678:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:0.0:1788572577.354683:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.354693:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.354702:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.354723:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572577.354792:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+259:2350:x1875453360681088:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 172us (1013us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.354803:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1164 00010000:00010000:3.0:1788572577.356862:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb165d36 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xac:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16b0 expref: 258 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.356888:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382407808:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572577.356913:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.356925:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.356940:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572577.357517:0:5839:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].78, 64 off14576 00010000:00010000:3.0:1788572577.358041:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 3/0,1 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358058:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358069:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358078:0:5839:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358085:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358092:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358121:0:5839:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788572577.358142:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358148:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.358298:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572577.358306:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.358330:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.363889:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572577.363902:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.363917:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.363944:0:5826:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:2.0:1788572577.363947:0:5826:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788572577.363952:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 1/0,0 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.363958:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661ab lrc: 0/0,0 mode: --/PW res: [0x200000402:0xac:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.363980:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572577.363982:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572577.363986:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572577.365660:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.365672:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365684:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365702:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xac:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:3.0:1788572577.365704:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572577.365709:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365717:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661a4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xac:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365750:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365758:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365765:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365770:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.365774:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365781:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365789:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.365791:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.365794:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.365800:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166196 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572577.365815:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006257e898 reply state 0000000009bf4d41 transno 4294971046 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.365825:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572577.366354:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+76:37341:x1875453360680832:12345-192.168.202.40@tcp:36:rm.0 Request processed in 24055us (24294us total) trans 4294971046 rc 0/0 00000100:00100000:3.0:1788572577.366364:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4971 00010000:00010000:2.0:1788572577.373288:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.373304:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.373311:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.373321:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.373326:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.373333:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.373350:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9626c4b10500) 00010000:00010000:2.0:1788572577.373352:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572577.373357:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.373364:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb16619d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.417841:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.418000:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360681344 00000100:00100000:3.0:1788572577.418046:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360681344 00000100:00100000:3.0:1788572577.418112:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4972 00000100:00100000:3.0:1788572577.418123:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37342:x1875453360681344:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:3.0:1788572577.418149:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572577.418182:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661b2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18b6 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.418317:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.418332:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.418356:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.418371:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.418377:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.418389:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788572577.418867:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5f:0x0], index 0: 0 00010000:00010000:3.0:1788572577.419051:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419061:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419069:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419104:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419111:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419117:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419122:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.419126:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419134:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419146:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.419149:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.419154:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419161:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661b9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572577.419172:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008db002e5 reply state 00000000735c59c0 transno 4294971047 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.419180:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788572577.419209:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661b2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 78 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419222:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 77 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.419229:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10900) 00010000:00010000:3.0:1788572577.419232:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572577.419235:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5800, rc 0) 00000100:00100000:3.0:1788572577.419305:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:37342:x1875453360681344:12345-192.168.202.40@tcp:101:dd.0 Request processed in 1184us (1472us total) trans 4294971047 rc 0/0 00000100:00100000:3.0:1788572577.419313:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4972 00000100:00100000:1.0:1788572577.427292:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:0.0:1788572577.427667:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.427667:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360681600 00000100:00100000:2.0:1788572577.427677:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360681600 00010000:00010000:0.0:1788572577.427678:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.427686:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.427691:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.427696:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.427703:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.427718:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9626c4b10800) 00010000:00010000:0.0:1788572577.427721:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.427726:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.427732:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1661c0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.427736:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1079 00000100:00100000:2.0:1788572577.427745:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:37342:x1875453360681600:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:2.0:1788572577.428156:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:37342:x1875453360681600:12345-192.168.202.40@tcp:35:dd.0 Request processed in 410us (873us total) trans 4294971048 rc 0/0 00000100:00100000:2.0:1788572577.428164:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1079 00000100:00100000:2.0:1788572577.528117:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.528224:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360681856 00000100:00100000:0.0:1788572577.528235:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360681856 00000100:00100000:0.0:1788572577.528302:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4973 00000100:00100000:0.0:1788572577.528313:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+258:37345:x1875453360681856:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572577.528338:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572577.528360:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1661d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18bd expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528467:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528483:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528496:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528586:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528595:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528603:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528724:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528734:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528742:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572577.528752:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb1:0x0]: returning size 0 00010000:00010000:0.0:1788572577.528919:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528933:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528940:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528945:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.528950:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528957:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528968:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572577.528971:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.528975:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528982:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1661dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.528995:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529144:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529153:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529159:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.529162:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529168:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529176:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9627c5160c00) 00010000:00010000:0.0:1788572577.529178:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.529182:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529187:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb1661e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529234:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1661d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 261 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529248:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 260 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.529256:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10900) 00010000:00010000:0.0:1788572577.529258:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572577.529261:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1400, rc 0) 00000100:00100000:0.0:1788572577.529321:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:37345:x1875453360681856:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1010us (1213us total) trans 4294971049 rc 0/0 00000100:00100000:0.0:1788572577.529328:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4973 00000100:00100000:0.0:1788572577.563349:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.563432:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360682112 00000100:00100000:3.0:1788572577.563460:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360682112 00000100:00100000:3.0:1788572577.563525:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4974 00000100:00100000:3.0:1788572577.563536:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:37345:x1875453360682112:12345-192.168.202.40@tcp:55:lfs.0 00010000:00010000:3.0:1788572577.563621:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.563647:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 77 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.563657:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 260 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.563666:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 5/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.563703:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 260 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.563717:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 260 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.563732:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382407936:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572577.563843:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 77 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.563854:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 77 pid: 9040 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.563863:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4b33800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382408064:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572577.563888:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102900 going to sleep for 11 seconds 00000100:00100000:3.0:1788572577.572549:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382407936:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572577.573676:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4b33800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382408064:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572577.573707:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.576372:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.576494:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360682240 00000100:00100000:0.0:1788572577.576505:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360682240 00010000:00010000:0.0:1788572577.576561:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 261 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.576603:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1165 00000100:00100000:0.0:1788572577.576613:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+261:2350:x1875453360682240:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572577.576630:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572577.576637:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 261 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.576652:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 261 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.576685:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1661ea lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18bd expref: 261 pid: 5838 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.576704:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10900) 00010000:00010000:0.0:1788572577.576708:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.576717:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.576725:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572577.576790:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:2350:x1875453360682240:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 178us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.576798:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1165 00000100:00100000:0.0:1788572577.583108:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.583259:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360682368 00000100:00100000:3.0:1788572577.583270:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360682368 00010000:00010000:3.0:1788572577.583316:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 78 pid: 9040 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.583364:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1166 00000100:00100000:3.0:1788572577.583376:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fad050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:2350:x1875453360682368:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.583392:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.583400:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 78 pid: 9040 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.583413:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 78 pid: 9040 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.583446:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1661c7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18b6 expref: 78 pid: 9040 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.583463:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10900) 00010000:00010000:3.0:1788572577.583468:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.583478:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.583485:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.584213:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572577.584291:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fad050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:2350:x1875453360682368:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 913us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.584301:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1166 00010000:00010000:0.0:1788572577.596478:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.596502:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599864:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599881:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599889:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599895:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.599900:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599918:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599936:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10900) 00010000:00010000:0.0:1788572577.599939:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572577.599945:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.599951:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb1661f8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.600074:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:37345:x1875453360682112:12345-192.168.202.40@tcp:55:lfs.0 Request processed in 36535us (36732us total) trans 4294971050 rc 0/0 00000100:00100000:0.0:1788572577.600085:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4974 00000100:00100000:0.0:1788572577.612746:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.613036:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360682624 00000100:00100000:1.0:1788572577.613045:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360682624 00000100:00100000:1.0:1788572577.614647:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1080 00000100:00100000:1.0:1788572577.614668:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+260:37345:x1875453360682624:12345-192.168.202.40@tcp:35:lfs.0 00000100:00100000:1.0:1788572577.614952:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+261:37345:x1875453360682624:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 291us (2219us total) trans 4294971051 rc 0/0 00000100:00100000:1.0:1788572577.614960:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1080 00000100:00100000:1.0:1788572577.642502:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.642548:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.642617:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360682880 00000100:00100000:3.0:1788572577.642629:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360682880 00000100:00100000:3.0:1788572577.642697:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 415 00000100:00100000:3.0:1788572577.642708:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+261:2347:x1875453360682880:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572577.642761:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572577.642784:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661ff lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18cb expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642850:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642865:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642878:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642893:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642899:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642906:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.642934:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360682752 00000100:00100000:2.0:1788572577.642938:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360682752 00010000:00010000:3.0:1788572577.642962:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.642974:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.642979:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4975 00010000:00010000:3.0:1788572577.642981:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.642986:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:37346:x1875453360682752:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:3.0:1788572577.642986:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.642991:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.642995:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572577.642998:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643028:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166214 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18c4 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643034:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.643037:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.643042:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643048:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166206 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643059:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643060:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643067:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643069:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643073:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643077:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643078:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.643082:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643087:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643089:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643094:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xccd5880f (ffff9626c4b10c00) 00010000:00010000:2.0:1788572577.643096:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643096:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.643099:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643102:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643105:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb16620d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xccd5880f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643132:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661ff lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18cb expref: 264 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643142:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.643144:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572577.643147:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1661ff lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18cb expref: 264 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.643156:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4000, rc -2) 00010000:00010000:2.0:1788572577.643173:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643179:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643196:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.643222:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:2347:x1875453360682880:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 517us (674us total) trans 0 rc -2/-2 00000004:00200000:2.0:1788572577.643229:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb1:0x0]: returning size 0 00000100:00100000:3.0:1788572577.643229:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 415 00000004:00200000:2.0:1788572577.643233:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572577.643251:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643262:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643268:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643274:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643279:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643284:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.643286:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643292:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643299:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572577.643301:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572577.643304:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643309:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16621b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643318:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643327:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643333:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643337:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.643340:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643346:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643353:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9627e3306a00) 00010000:00010000:2.0:1788572577.643356:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572577.643359:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643364:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166222 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643399:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166214 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 263 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643412:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 262 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.643419:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306d00) 00010000:00010000:2.0:1788572577.643421:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572577.643424:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aea00, rc 0) 00000100:00100000:2.0:1788572577.643468:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+262:37346:x1875453360682752:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 482us (977us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572577.643475:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4975 00000100:00100000:1.0:1788572577.651210:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.651287:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360683008 00000100:00100000:0.0:1788572577.651298:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360683008 00000100:00080000:1.0:1788572577.651350:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788572577.651370:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1457 00000100:00100000:0.0:1788572577.651380:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360683008:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572577.651399:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572577.651419:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18d2 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572577.651459:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572577.651478:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=925s v=(0 0) 00010000:00010000:0.0:1788572577.651483:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572577.651493:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 75 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.651524:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 75 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.651536:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1400, rc 0) 00000100:00100000:0.0:1788572577.651602:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2350:x1875453360683008:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 223us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572577.651610:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1457 00000100:00100000:3.0:1788572577.661129:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572577.661447:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360683136 00000100:00100000:2.0:1788572577.661458:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360683136 00000100:00100000:2.0:1788572577.661513:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4976 00000100:00100000:2.0:1788572577.661523:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+262:37346:x1875453360683136:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572577.661544:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572577.661565:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166237 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18d9 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.661655:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.661670:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.661683:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572577.661726:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572577.661730:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572577.661749:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.661770:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166237 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 264 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.661784:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 263 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.661791:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572577.661793:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572577.661795:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae600, rc 0) 00000100:00100000:2.0:1788572577.661855:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:37346:x1875453360683136:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 332us (736us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572577.661862:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4976 00000100:00100000:3.0:1788572577.668811:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.670091:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360683392 00000100:00100000:1.0:1788572577.670104:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360683392 00000100:00100000:1.0:1788572577.670277:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4977 00000100:00100000:1.0:1788572577.670292:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+263:37346:x1875453360683392:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572577.670315:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572577.670338:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166245 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18e0 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.670469:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16624c lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.670485:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16624c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.670499:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16624c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572577.670522:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb1:0x0]: returning size 0 00010000:00010000:1.0:1788572577.670671:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166245 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e0 expref: 266 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.670690:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16624c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e0 expref: 265 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.670699:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306d00) 00010000:00010000:1.0:1788572577.670702:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1788572577.670706:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647200, rc 0) 00000100:00100000:1.0:1788572577.670776:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+265:37346:x1875453360683392:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 487us (1976us total) trans 4294971052 rc 0/0 00000100:00100000:1.0:1788572577.670784:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4977 00000100:00100000:0.0:1788572577.678776:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.678980:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360683520 00000100:00100000:1.0:1788572577.678993:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360683520 00000100:00100000:1.0:1788572577.679092:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1458 00000100:00100000:1.0:1788572577.679102:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2349:x1875453360683520:12345-192.168.202.40@tcp:1:lfs.0 00010000:00010000:1.0:1788572577.679146:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.679169:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 75 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.679177:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572577.679182:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572577.679198:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 4/0,1 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.679240:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 75 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.679256:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 75 pid: 6681 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.679271:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382408192:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572577.679357:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ffe82f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572577.694287:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6679:1875453382408192:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572577.694322:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.694963:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.695778:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360683648 00000100:00100000:1.0:1788572577.695791:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360683648 00010000:00010000:1.0:1788572577.695835:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 76 pid: 6681 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572577.695888:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1167 00000100:00100000:1.0:1788572577.695899:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2350:x1875453360683648:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572577.695925:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572577.695951:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 76 pid: 6681 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.695965:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 76 pid: 6681 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.695988:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f198336f/0x85ace67feb166230 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d2 expref: 76 pid: 6681 timeout: 1025 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.696092:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572577.696097:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572577.696105:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.696117:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.696156:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572577.696215:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2350:x1875453360683648:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 317us (1254us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572577.696222:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1167 00010000:00010000:2.0:1788572577.697234:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697250:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697282:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697291:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697298:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697305:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572577.697309:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697324:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697341:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.697348:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb16625a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.697460:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2349:x1875453360683520:12345-192.168.202.40@tcp:1:lfs.0 Request processed in 18356us (18833us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572577.697468:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1458 00000100:00100000:0.0:1788572577.700114:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.700205:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360683904 00000100:00100000:1.0:1788572577.700216:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360683904 00000100:00100000:1.0:1788572577.700269:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1081 00000100:00100000:1.0:1788572577.700279:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+265:37346:x1875453360683904:12345-192.168.202.40@tcp:35:lfs.0 00000004:00010000:1.0:1788572577.700346:0:5840:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0xb1:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16624c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e0 expref: 265 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572577.706068:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971053 00000001:00080000:2.0:1788572577.706888:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971054 00010000:00010000:2.0:1788572577.706957:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.706979:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 268 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.706988:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.707051:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 268 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.707069:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 268 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.707085:0:5840:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755b100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5840:1875453382408320:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572577.707175:0:5840:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572577.711222:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.711386:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360684032 00000100:00100000:3.0:1788572577.711396:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360684032 00010000:00010000:3.0:1788572577.711454:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 269 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.711503:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1168 00000100:00100000:3.0:1788572577.711514:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:2350:x1875453360684032:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.711531:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.711538:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 269 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.711562:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 269 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.711586:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306d00) 00010000:00010000:3.0:1788572577.711591:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.711601:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.711609:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.711808:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572577.712270:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166229 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18c4 expref: 269 pid: 22536 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.712290:0:5840:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755b100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5840:1875453382408320:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572577.712316:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.712326:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.712332:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572577.712490:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971055 00000001:00080000:2.0:1788572577.713157:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971056 00000001:00080000:2.0:1788572577.713230:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971057 00000100:00100000:2.0:1788572577.713337:0:5840:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e755b100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5840:1875453382408448:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1788572577.713410:0:5840:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572577.713457:0:5840:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c94c6300 going to sleep for 11 seconds 00000100:00100000:2.0:1788572577.714154:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382408448 00000100:00100000:2.0:1788572577.714163:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382408448 00000100:00100000:2.0:1788572577.714234:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:2.0:1788572577.714243:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5840:x1875453382408448:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00080000:1.0:1788572577.714345:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572577.714577:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5840:x1875453382408448:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 334us (1165us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572577.714586:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1788572577.714789:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:2350:x1875453360684032:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3274us (3574us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.714800:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1168 00000100:00100000:1.0:1788572577.715947:0:5840:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e755b100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5840:1875453382408448:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:0.0:1788572577.716199:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971058 00010000:00010000:0.0:1788572577.716820:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.716833:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.716842:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.716849:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.716853:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.716862:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.716878:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306d00) 00010000:00010000:0.0:1788572577.716881:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1788572577.716887:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.716894:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166261 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572577.721497:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0xd2:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1788572577.723850:0:5840:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].81, 64 off14768 00000001:00080000:0.0:1788572577.723887:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971059 00010000:00010000:0.0:1788572577.723982:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306d00) 00010000:00010000:0.0:1788572577.723988:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1788572577.724016:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb16624c lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e0 expref: 272 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.724278:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:37346:x1875453360683904:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 23997us (24169us total) trans 4294971060 rc 0/0 00000100:00100000:0.0:1788572577.724291:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1081 00000020:00080000:0.0:1788572577.724298:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:0.0:1788572577.794247:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.794344:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360684160 00000100:00100000:1.0:1788572577.794356:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360684160 00000100:00100000:1.0:1788572577.794439:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4978 00000100:00100000:1.0:1788572577.794452:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:37350:x1875453360684160:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:1.0:1788572577.794480:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572577.794503:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166268 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18e7 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.794592:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16626f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.794609:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16626f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.794633:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16626f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572577.794669:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:1.0:1788572577.794673:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572577.794690:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16626f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.794719:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166268 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e7 expref: 79 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.794739:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16626f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e7 expref: 78 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572577.794748:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572577.794751:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572577.794755:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646c00, rc 0) 00000100:00100000:1.0:1788572577.794819:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:37350:x1875453360684160:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 371us (586us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572577.794827:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4978 00000100:00100000:2.0:1788572577.841212:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.841316:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360684416 00000100:00100000:0.0:1788572577.841327:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360684416 00000100:00100000:0.0:1788572577.841401:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4979 00000100:00100000:0.0:1788572577.841413:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:37351:x1875453360684416:12345-192.168.202.40@tcp:36:rm.0 00010000:00010000:0.0:1788572577.841498:0:5838:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788572577.841529:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54e400/0x85ace67feb165e24 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f176d expref: 78 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.841563:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:0.0:1788572577.841577:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16626f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18e7 expref: 77 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.841586:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572577.841590:0:5838:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572577.841614:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.841642:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 270 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.841650:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.841683:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 270 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.841696:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 270 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.841709:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d0fb0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382408576:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572577.841972:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572577.857847:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d0fb0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382408576:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572577.857882:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.858326:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.865313:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360684544 00000100:00100000:3.0:1788572577.865328:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360684544 00010000:00010000:3.0:1788572577.865398:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 271 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.865461:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1169 00000100:00100000:3.0:1788572577.865473:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:2350:x1875453360684544:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.865492:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.865499:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 271 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.865513:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 271 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.865537:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16623e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18d9 expref: 271 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.865568:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572577.865574:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.865584:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.865592:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.865848:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572577.865936:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+270:2350:x1875453360684544:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 463us (7611us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.865945:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1169 00010000:00010000:2.0:1788572577.866708:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866720:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866751:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866764:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866786:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866902:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866915:0:5838:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 269 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866923:0:5838:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866963:0:5838:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 269 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.866978:0:5838:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 269 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.866994:0:5838:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382408704:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572577.867077:0:5838:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572577.875066:0:5838:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5838:1875453382408704:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572577.875101:0:5838:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.883216:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572577.884484:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360684672 00000100:00100000:3.0:1788572577.884499:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360684672 00010000:00010000:3.0:1788572577.884562:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 270 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.884599:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1170 00000100:00100000:3.0:1788572577.884610:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+270:2350:x1875453360684672:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572577.884635:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572577.884644:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 270 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.884658:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 270 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.884680:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb165e6a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xad:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1782 expref: 270 pid: 23058 timeout: 1025 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.884702:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:3.0:1788572577.884709:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.884722:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.884732:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.884957:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572577.885078:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:2350:x1875453360684672:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 468us (1874us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572577.885087:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1170 00010000:00010000:2.0:1788572577.886987:0:5838:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.887021:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572577.899130:0:5838:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].82, 64 off14832 00010000:00010000:2.0:1788572577.899501:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.899524:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.899537:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900037:0:5838:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900045:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900053:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900083:0:5838:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788572577.900109:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900116:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900121:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900126:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.900131:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900139:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900152:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:2.0:1788572577.900155:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572577.900160:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.900166:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166284 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.902372:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572577.902390:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.902403:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572577.902408:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.902418:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.902450:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xad:0x0].0x0 (ffff9627c5160800) 00010000:00010000:0.0:1788572577.902455:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572577.902459:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 1/0,0 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.902465:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16628b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xad:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902560:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902569:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902576:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902581:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.902585:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902592:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902602:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572577.902605:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572577.902608:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.902614:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166276 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.910582:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572577.910590:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572577.910594:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:2.0:1788572577.910774:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000320aceae reply state 00000000bcba912c transno 4294971061 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572577.910792:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572577.911039:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e755b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+78:37351:x1875453360684416:12345-192.168.202.40@tcp:36:rm.0 Request processed in 69622us (69832us total) trans 4294971061 rc 0/0 00000100:00100000:2.0:1788572577.911049:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4979 00010000:00010000:3.0:1788572577.915405:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572577.920042:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.920060:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.920073:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.920080:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572577.920085:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.920094:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.920112:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627e3306a00) 00010000:00010000:2.0:1788572577.920115:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572577.920120:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572577.920127:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16627d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.965859:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572577.971454:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360684928 00000100:00100000:0.0:1788572577.971470:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360684928 00000100:00100000:0.0:1788572577.971600:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4980 00000100:00100000:0.0:1788572577.971613:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+77:37352:x1875453360684928:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:0.0:1788572577.971648:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572577.971672:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb166292 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18ee expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.971815:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.971830:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.971843:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.971857:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.971865:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.971872:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572577.972432:0:5837:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x5f:0x0], index 1: 0 00010000:00010000:0.0:1788572577.972606:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972616:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972624:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972662:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972670:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972677:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972682:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572577.972687:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972694:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972707:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572577.972710:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572577.972715:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972722:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166299 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572577.972739:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c0313077 reply state 0000000045ed08bb transno 4294971062 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572577.972749:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1788572577.972785:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb166292 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 80 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972797:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 79 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572577.972804:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:0.0:1788572577.972806:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572577.972808:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1e00, rc 0) 00000100:00100000:0.0:1788572577.972903:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:37352:x1875453360684928:12345-192.168.202.40@tcp:101:dd.0 Request processed in 1291us (7053us total) trans 4294971062 rc 0/0 00000100:00100000:0.0:1788572577.972911:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4980 00010000:00010000:3.0:1788572577.980164:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.980184:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.980191:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.980198:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572577.980202:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.980211:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.980230:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627c5160800) 00010000:00010000:3.0:1788572577.980233:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572577.980238:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572577.980245:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb1662a0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572577.981478:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572577.983458:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360685184 00000100:00100000:1.0:1788572577.983474:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360685184 00000100:00100000:1.0:1788572577.983552:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1082 00000100:00100000:1.0:1788572577.983563:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:37352:x1875453360685184:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:1.0:1788572577.985736:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:37352:x1875453360685184:12345-192.168.202.40@tcp:35:dd.0 Request processed in 2164us (4257us total) trans 4294971063 rc 0/0 00000100:00100000:1.0:1788572577.985752:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1082 00000100:00100000:3.0:1788572578.106562:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000004:00080000:2.0:1788572578.106626:0:6873:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 926175621148ns, next at 927175617944ns, rc = 0 00000100:00100000:3.0:1788572578.111137:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382408832:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788572578.111236:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572578.111575:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382408832 00000100:00100000:0.0:1788572578.111587:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382408832 00000100:00100000:0.0:1788572578.111659:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:0.0:1788572578.111673:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382408832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788572578.111721:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 197197824 pending 0 free 3855613952 avail 3853516800 00000100:00100000:0.0:1788572578.111803:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1875453382408832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 134us (567us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.111810:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:1.0:1788572578.116433:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382408832:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788572578.126068:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.126566:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360685440 00000100:00100000:1.0:1788572578.126579:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360685440 00000100:00100000:1.0:1788572578.126657:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4981 00000100:00100000:1.0:1788572578.126669:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+269:37355:x1875453360685440:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572578.126697:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.126724:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1662b5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18f5 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127148:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127165:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127178:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127194:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127202:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127209:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127717:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127731:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.127741:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572578.127753:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb2:0x0]: returning size 0 00010000:00010000:1.0:1788572578.128095:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128104:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128112:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128118:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.128123:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128829:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128846:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.128849:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.128855:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128862:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.128879:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129117:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129124:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129140:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.129144:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129151:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129161:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572578.129164:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.129167:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129181:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129243:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1662b5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 272 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129272:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 271 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.129280:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:1.0:1788572578.129283:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572578.129286:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647200, rc 0) 00000100:00100000:1.0:1788572578.129376:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:37355:x1875453360685440:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 2708us (3320us total) trans 4294971064 rc 0/0 00000100:00100000:1.0:1788572578.129386:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4981 00000100:00100000:3.0:1788572578.133601:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.133902:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360685696 00000100:00100000:2.0:1788572578.133913:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360685696 00000100:00100000:2.0:1788572578.133987:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4982 00000100:00100000:2.0:1788572578.133999:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:37355:x1875453360685696:12345-192.168.202.40@tcp:55:lfs.0 00010000:00010000:2.0:1788572578.134108:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.134141:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 79 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.134156:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 271 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.134165:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 5/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.134207:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 271 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.134221:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 271 pid: 22536 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.134238:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382408960:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.134849:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 79 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.134862:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 79 pid: 5837 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.134874:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382409088:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572578.134913:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572578.148274:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.148403:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.148475:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360685824 00000100:00100000:1.0:1788572578.148486:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360685824 00010000:00010000:1.0:1788572578.148532:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 272 pid: 22536 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.148575:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360685952 00000100:00100000:1.0:1788572578.148578:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360685952 00010000:00010000:1.0:1788572578.148606:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 80 pid: 5837 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.148626:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1171 00000100:00100000:1.0:1788572578.148636:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:2350:x1875453360685824:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.148652:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.148659:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 272 pid: 22536 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148673:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 272 pid: 22536 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148696:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:1.0:1788572578.148702:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148711:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148718:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572578.148769:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:2350:x1875453360685824:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 134us (522us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.148776:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1171 00000100:00100000:1.0:1788572578.148797:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1172 00000100:00100000:1.0:1788572578.148804:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:2350:x1875453360685952:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.148813:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.148818:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 80 pid: 5837 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148829:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 80 pid: 5837 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148842:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:1.0:1788572578.148845:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148854:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148862:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.148881:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572578.148909:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:2350:x1875453360685952:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 105us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.148916:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1172 00000100:00080000:1.0:1788572578.148935:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1788572578.154801:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1662ca lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18f5 expref: 271 pid: 22536 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.154837:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382408960:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.154858:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb1662a7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18ee expref: 79 pid: 5837 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.154871:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382409088:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.154912:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.154923:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.154928:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155239:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155249:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155257:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155264:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.155270:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155279:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155297:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160c00) 00010000:00010000:2.0:1788572578.155300:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572578.155305:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.155313:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1662d8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.155406:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff5ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:37355:x1875453360685696:12345-192.168.202.40@tcp:55:lfs.0 Request processed in 21408us (21814us total) trans 4294971065 rc 0/0 00000100:00100000:2.0:1788572578.155426:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4982 00000100:00100000:3.0:1788572578.159411:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.159473:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360686208 00000100:00100000:0.0:1788572578.159494:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360686208 00000100:00100000:0.0:1788572578.159663:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1083 00000100:00100000:0.0:1788572578.159675:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13d450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+271:37355:x1875453360686208:12345-192.168.202.40@tcp:35:lfs.0 00000100:00100000:0.0:1788572578.159949:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13d450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:37355:x1875453360686208:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 275us (549us total) trans 4294971066 rc 0/0 00000100:00100000:0.0:1788572578.159957:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1083 00000100:00100000:3.0:1788572578.184555:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.184648:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360686336 00000100:00100000:1.0:1788572578.184659:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360686336 00000100:00100000:1.0:1788572578.184765:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4983 00000100:00100000:1.0:1788572578.184777:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+272:37356:x1875453360686336:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572578.184803:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.184827:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1662df lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f18fc expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.184916:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.184931:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.184943:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.184957:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.184964:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.184971:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185233:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185244:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185253:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572578.185287:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb2:0x0]: returning size 0 00000004:00200000:1.0:1788572578.185292:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572578.185311:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185324:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185333:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185340:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185348:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185353:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.185358:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185366:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185378:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.185381:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.185387:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185398:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1662e6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185411:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185428:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185435:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185440:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.185443:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185451:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185459:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627ff5abd00) 00010000:00010000:1.0:1788572578.185461:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.185465:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185471:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1662ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185505:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb1662df lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 274 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185526:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 273 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.185533:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5ab300) 00010000:00010000:1.0:1788572578.185536:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572578.185538:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647c00, rc 0) 00000100:00100000:1.0:1788572578.185607:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb30e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+273:37356:x1875453360686336:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 832us (1064us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.185615:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4983 00000100:00100000:3.0:1788572578.189751:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.189822:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360686464 00000100:00100000:1.0:1788572578.189832:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360686464 00000100:00080000:2.0:1788572578.189975:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572578.190322:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1459 00000100:00100000:1.0:1788572578.190337:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2347:x1875453360686464:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572578.190357:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.190381:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1903 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572578.190410:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1788572578.190444:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=926s v=(0 0) 00010000:00010000:1.0:1788572578.190449:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572578.190459:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 86 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.190480:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 86 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.190500:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647800, rc 0) 00000100:00100000:1.0:1788572578.190564:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2347:x1875453360686464:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 232us (824us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.196058:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1459 00000100:00100000:3.0:1788572578.196324:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.196422:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360686592 00000100:00100000:1.0:1788572578.196431:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360686592 00000100:00100000:1.0:1788572578.196531:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4984 00000100:00100000:1.0:1788572578.196542:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+273:37356:x1875453360686592:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572578.196566:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.196587:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166302 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f190a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.196663:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.196678:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.196691:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572578.196728:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:1.0:1788572578.196732:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572578.196749:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.196777:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166302 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 275 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.196795:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 274 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.196804:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.196807:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572578.196811:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646800, rc 0) 00000100:00100000:1.0:1788572578.196899:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+274:37356:x1875453360686592:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 358us (576us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.196906:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4984 00000100:00100000:2.0:1788572578.202872:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.203115:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360686848 00000100:00100000:1.0:1788572578.203130:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360686848 00000100:00100000:1.0:1788572578.203206:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4985 00000100:00100000:1.0:1788572578.203220:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+274:37356:x1875453360686848:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572578.203248:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.203277:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166310 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1911 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.203402:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb166317 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.203428:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb166317 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.203444:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb166317 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572578.203457:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb2:0x0]: returning size 0 00010000:00010000:1.0:1788572578.203616:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166310 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1911 expref: 277 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.203640:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb166317 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1911 expref: 276 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.203650:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5ab300) 00010000:00010000:1.0:1788572578.203654:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1788572578.203658:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646400, rc 0) 00000100:00100000:1.0:1788572578.203735:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+276:37356:x1875453360686848:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 517us (885us total) trans 4294971067 rc 0/0 00000100:00100000:1.0:1788572578.203744:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4985 00000100:00100000:3.0:1788572578.207829:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.207940:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360686976 00000100:00100000:2.0:1788572578.207951:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360686976 00000100:00100000:2.0:1788572578.208045:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1460 00000100:00100000:2.0:1788572578.208055:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2350:x1875453360686976:12345-192.168.202.40@tcp:1:lfs.0 00010000:00010000:2.0:1788572578.208102:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.208122:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 86 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.208131:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572578.208136:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572578.208141:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.208187:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 86 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.208207:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 86 pid: 6682 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.208221:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382409216:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572578.208305:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572578.212533:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:10416:1875453382409216:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.212571:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.214460:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.214773:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360687104 00000100:00100000:1.0:1788572578.214784:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360687104 00010000:00010000:1.0:1788572578.214826:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 87 pid: 6682 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.214873:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1173 00000100:00100000:1.0:1788572578.214892:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+87:2350:x1875453360687104:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.214909:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.214932:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 87 pid: 6682 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.214946:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 87 pid: 6682 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.214968:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077621369/0x85ace67feb1662fb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1903 expref: 87 pid: 6682 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.214988:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572578.214992:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572578.214998:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.215357:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.215445:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572578.215513:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2350:x1875453360687104:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 631us (1061us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.215521:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1173 00010000:00010000:2.0:1788572578.215643:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215653:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215683:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215692:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215700:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215712:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572578.215716:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215726:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215740:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.215748:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001a63e273/0x85ace67feb166325 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.215797:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2350:x1875453360686976:12345-192.168.202.40@tcp:1:lfs.0 Request processed in 7744us (7978us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.215805:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1460 00000100:00100000:3.0:1788572578.219779:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.219858:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360687360 00000100:00100000:2.0:1788572578.219869:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360687360 00000100:00100000:2.0:1788572578.219949:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1084 00000100:00100000:2.0:1788572578.219960:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+276:37356:x1875453360687360:12345-192.168.202.40@tcp:35:lfs.0 00000004:00010000:2.0:1788572578.220062:0:5841:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0xb2:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb166317 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1911 expref: 276 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572578.220290:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971068 00000001:00080000:2.0:1788572578.220636:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971069 00010000:00010000:2.0:1788572578.220699:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.220714:0:5841:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 279 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.220723:0:5841:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.220758:0:5841:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 279 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.220772:0:5841:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 279 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.220787:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5841:1875453382409344:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572578.220879:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d600 going to sleep for 11 seconds 00000100:00100000:2.0:1788572578.224584:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5841:1875453382409344:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.224616:0:5841:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.231455:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.238344:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360687488 00000100:00100000:1.0:1788572578.238362:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360687488 00010000:00010000:1.0:1788572578.238424:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 280 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.241154:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1174 00000100:00100000:2.0:1788572578.241194:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:2349:x1875453360687488:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572578.241221:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572578.241234:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 280 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.241257:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 280 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.241280:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1662f4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f18fc expref: 280 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.241296:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5ab300) 00010000:00010000:2.0:1788572578.241302:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.241312:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.241320:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.241357:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572578.241434:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+279:2349:x1875453360687488:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 254us (9989us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.241442:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1174 00010000:00010000:1.0:1788572578.242272:0:5841:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.242283:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572578.242429:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971070 00000001:00080000:1.0:1788572578.242662:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971071 00000001:00080000:1.0:1788572578.242729:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971072 00000100:00100000:1.0:1788572578.242820:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfae7480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5841:1875453382409472:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1788572578.242882:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572578.242936:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572578.243405:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382409472 00000100:00100000:0.0:1788572578.243423:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382409472 00000100:00100000:0.0:1788572578.243487:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:0.0:1788572578.243497:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5841:x1875453382409472:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1788572578.243586:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5841:x1875453382409472:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 90us (703us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.243594:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:0.0:1788572578.243639:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfae7480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5841:1875453382409472:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:0.0:1788572578.243698:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971073 00010000:00010000:0.0:1788572578.243732:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.243741:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.243748:0:5841:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.243753:0:5841:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5af200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.243758:0:5841:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.243766:0:5841:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.243780:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5ab300) 00010000:00010000:0.0:1788572578.243783:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1788572578.243788:0:5841:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.243794:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16632c lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572578.244285:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0xd3:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1788572578.244360:0:5841:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].79, 64 off14640 00000001:00080000:0.0:1788572578.244373:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971074 00010000:00010000:0.0:1788572578.244403:0:5841:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5ab300) 00010000:00010000:0.0:1788572578.244407:0:5841:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1788572578.244424:0:5841:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb166317 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1911 expref: 283 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.244559:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:37356:x1875453360687360:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 24601us (24791us total) trans 4294971075 rc 0/0 00000100:00100000:0.0:1788572578.244568:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1084 00000020:00080000:0.0:1788572578.244572:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:3.0:1788572578.345768:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.346489:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360687616 00000100:00100000:0.0:1788572578.346501:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360687616 00000100:00100000:0.0:1788572578.346621:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4986 00000100:00100000:0.0:1788572578.346632:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:37360:x1875453360687616:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:0.0:1788572578.346657:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572578.346681:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166333 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1918 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.346769:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16633a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.346784:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16633a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.346796:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16633a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572578.346837:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572578.346841:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572578.346876:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16633a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.346906:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166333 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1918 expref: 81 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.346925:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16633a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1918 expref: 80 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.346934:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.346948:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572578.346952:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1800, rc 0) 00000100:00100000:0.0:1788572578.347080:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:37360:x1875453360687616:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 449us (1325us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.347088:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4986 00000100:00100000:1.0:1788572578.373225:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.373283:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.373499:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360687744 00000100:00100000:0.0:1788572578.373512:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360687744 00000100:00100000:0.0:1788572578.373579:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 418 00000100:00100000:0.0:1788572578.373590:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:2349:x1875453360687744:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572578.373628:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572578.373649:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166341 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f191f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373714:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373732:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373742:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373759:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373765:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373771:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373828:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373835:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373841:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373846:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.373850:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373858:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373869:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.373873:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.373878:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373884:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166348 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373899:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373904:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373909:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373914:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.373916:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373922:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373929:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xb3dc5f41 (ffff9627c5160a00) 00010000:00010000:0.0:1788572578.373932:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.373935:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373941:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16634f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373973:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166341 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f191f expref: 81 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373983:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.373985:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572578.373988:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1000/0x85ace67feb166341 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f191f expref: 81 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.373997:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1000, rc -2) 00000100:00100000:0.0:1788572578.374107:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfae7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:2349:x1875453360687744:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 518us (894us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572578.374115:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 418 00000100:00100000:2.0:1788572578.379445:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360688000 00000100:00100000:2.0:1788572578.379463:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360688000 00000100:00100000:2.0:1788572578.379554:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4987 00000100:00100000:2.0:1788572578.379565:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:37361:x1875453360688000:12345-192.168.202.40@tcp:36:rm.0 00010000:00010000:2.0:1788572578.379626:0:22536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1788572578.379654:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb165ee1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17b3 expref: 80 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.379679:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:2.0:1788572578.379692:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16633a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1918 expref: 79 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.379702:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572578.379706:0:22536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572578.379732:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.379745:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 281 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.379754:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.379791:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 281 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.379805:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 281 pid: 9040 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.379820:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382409600:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572578.379934:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d600 going to sleep for 11 seconds 00000100:00100000:3.0:1788572578.391254:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.391387:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382409600:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572578.391434:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.391470:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360688128 00000100:00100000:0.0:1788572578.391478:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360688128 00010000:00010000:0.0:1788572578.391519:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 282 pid: 9040 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.391553:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1175 00000100:00100000:0.0:1788572578.391562:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:2349:x1875453360688128:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572578.391578:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572578.391585:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 282 pid: 9040 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391599:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 282 pid: 9040 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391642:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb166309 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f190a expref: 282 pid: 9040 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391659:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.391664:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391675:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391683:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391707:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572578.391758:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+281:2349:x1875453360688128:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 196us (514us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.391765:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1175 00010000:00010000:0.0:1788572578.391788:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391795:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391818:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391829:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391837:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391967:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391978:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 280 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.391986:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.392041:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 280 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.392055:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 280 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.392070:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7bc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382409728:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572578.392119:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572578.395814:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7bc6680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382409728:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572578.395866:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.396338:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.396904:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360688256 00000100:00100000:1.0:1788572578.396913:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360688256 00010000:00010000:1.0:1788572578.396953:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 281 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.396995:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1176 00000100:00100000:1.0:1788572578.397027:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+281:2350:x1875453360688256:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.397043:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.397051:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 281 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.397065:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 281 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.397086:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb54f200/0x85ace67feb165f0b lrc: 0/0,0 mode: --/PR res: [0x200000402:0xae:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17c1 expref: 281 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.397104:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:1.0:1788572578.397110:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.397120:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.397132:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.397155:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572578.397252:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397260:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1788572578.397825:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].80, 64 off14704 00000004:00080000:0.0:1788572578.397835:0:22536:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 926466830597ns, next at 927466827750ns, rc = 0 00010000:00010000:0.0:1788572578.397901:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397914:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397924:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397931:0:22536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397937:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397944:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397961:0:22536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788572578.397982:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397988:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.397994:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398000:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.398030:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398042:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398052:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:0.0:1788572578.398055:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572578.398058:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398064:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166364 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398084:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398090:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398096:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398101:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.398104:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398110:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398117:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.398120:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.398123:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.398133:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166356 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572578.398142:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008f27e126 reply state 00000000b9bc33c4 transno 4294971076 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.398150:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572578.398333:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:2350:x1875453360688256:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 1326us (1994us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.398344:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1176 00000100:00100000:0.0:1788572578.398996:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+80:37361:x1875453360688000:12345-192.168.202.40@tcp:36:rm.0 Request processed in 19432us (25709us total) trans 4294971076 rc 0/0 00000100:00100000:0.0:1788572578.399042:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4987 00010000:00010000:3.0:1788572578.399147:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572578.399155:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.399175:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572578.399178:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.399186:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.399198:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xae:0x0].0x0 (ffff9627ff5ab600) 00010000:00010000:3.0:1788572578.399200:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572578.399204:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 1/0,0 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.399211:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16636b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xae:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.399232:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572578.399233:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572578.399235:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572578.403969:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.403996:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.404027:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.404034:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.404039:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.404048:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.404064:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627c5160300) 00010000:00010000:0.0:1788572578.404067:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.404073:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.404083:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16635d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.416394:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.416637:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360688384 00000100:00100000:1.0:1788572578.416649:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360688384 00000100:00100000:1.0:1788572578.416714:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 419 00000100:00100000:1.0:1788572578.416725:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:2350:x1875453360688384:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572578.416770:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.416797:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166372 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1950 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.416868:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.416896:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.416909:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.416926:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.416933:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.416943:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442223:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442242:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442250:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442256:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.442262:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442270:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442290:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572578.442293:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.442299:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442305:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647600/0x85ace67feb166379 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442324:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442330:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442347:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442353:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.442355:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442362:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442369:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x8799b2a3 (ffff9627ff5abd00) 00010000:00010000:3.0:1788572578.442371:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.442374:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442379:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166380 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa2:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442439:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166372 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1950 expref: 281 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442450:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa2:0x0].0x0 (ffff9627fb1e7500) 00010000:00010000:3.0:1788572578.442452:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572578.442456:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166372 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xa2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1950 expref: 281 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.442466:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647800, rc -2) 00000100:00100000:3.0:1788572578.442565:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:2350:x1875453360688384:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 25840us (26179us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572578.442576:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 419 00000100:00100000:3.0:1788572578.446991:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.447108:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360688640 00000100:00100000:1.0:1788572578.447120:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360688640 00000100:00100000:1.0:1788572578.447197:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4988 00000100:00100000:1.0:1788572578.447208:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+79:37364:x1875453360688640:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:1.0:1788572578.447235:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.447258:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166387 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1957 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.447422:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.447437:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.447449:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.447464:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.447471:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.447478:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572578.448000:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x60:0x0], index 0: 0 00010000:00010000:1.0:1788572578.448222:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448231:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448239:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448272:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448279:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448285:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448291:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.448295:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448303:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448314:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.448317:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.448322:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448328:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16638e lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572578.448344:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b8b6145d reply state 000000009dc51c6a transno 4294971077 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.448352:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1788572578.448387:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166387 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 82 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448401:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 81 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.448421:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572578.448423:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572578.448426:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646a00, rc 0) 00000100:00100000:1.0:1788572578.448500:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:37364:x1875453360688640:12345-192.168.202.40@tcp:101:dd.0 Request processed in 1294us (1519us total) trans 4294971077 rc 0/0 00000100:00100000:1.0:1788572578.448508:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4988 00010000:00010000:3.0:1788572578.460348:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.460367:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.460375:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.460382:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.460387:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.460396:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.460423:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627ff5abb00) 00010000:00010000:3.0:1788572578.460427:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.460431:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.460438:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166395 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.462808:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.463056:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360688896 00000100:00100000:0.0:1788572578.463068:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360688896 00000100:00100000:0.0:1788572578.463128:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1085 00000100:00100000:0.0:1788572578.463138:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:37364:x1875453360688896:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:0.0:1788572578.463553:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:37364:x1875453360688896:12345-192.168.202.40@tcp:35:dd.0 Request processed in 415us (754us total) trans 4294971078 rc 0/0 00000100:00100000:0.0:1788572578.463560:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1085 00000100:00100000:0.0:1788572578.528937:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.529685:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360689024 00000100:00100000:1.0:1788572578.529696:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360689024 00000100:00100000:1.0:1788572578.529761:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 420 00000100:00100000:1.0:1788572578.529770:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:2349:x1875453360689024:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572578.529807:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.529825:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1663aa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f195e expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529880:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529899:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529911:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529925:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529933:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529939:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529993:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.529999:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530053:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530058:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.530063:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530071:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530084:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.530088:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.530092:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530097:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1663b1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530110:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530115:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530120:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530124:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.530127:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530133:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530139:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xb3dc5f41 (ffff9627ff5ab800) 00010000:00010000:1.0:1788572578.530141:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.530144:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530150:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb1663b8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530183:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1663aa lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f195e expref: 83 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530193:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.530196:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572578.530199:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1663aa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f195e expref: 83 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.530274:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646c00, rc -2) 00000100:00100000:1.0:1788572578.532330:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:2349:x1875453360689024:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2548us (3388us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572578.532355:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 420 00000100:00100000:1.0:1788572578.590320:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.590549:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360689280 00000100:00100000:3.0:1788572578.590562:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360689280 00000100:00100000:3.0:1788572578.590642:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4989 00000100:00100000:3.0:1788572578.590654:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+280:37369:x1875453360689280:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:3.0:1788572578.590679:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572578.590703:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1663bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f198f expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.590788:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.590804:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.590816:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.590831:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.590839:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.590845:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.591158:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.591172:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.591179:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572578.591189:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00010000:00010000:2.0:1788572578.591958:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.591969:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.591978:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.591984:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.591989:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.591997:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592039:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572578.592043:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.592048:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592054:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1663c6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592068:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592074:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592080:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592085:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.592088:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592099:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592107:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9626c4b10c00) 00010000:00010000:2.0:1788572578.592109:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.592112:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592118:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1663cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592173:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1663bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 283 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592188:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 282 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.592195:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:2.0:1788572578.592198:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572578.592200:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5000, rc 0) 00000100:00100000:2.0:1788572578.592666:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:37369:x1875453360689280:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 2011us (2354us total) trans 4294971079 rc 0/0 00000100:00100000:2.0:1788572578.592677:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4989 00000100:00100000:1.0:1788572578.609692:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.609816:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360689536 00000100:00100000:3.0:1788572578.609829:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360689536 00000100:00100000:3.0:1788572578.610042:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4990 00000100:00100000:3.0:1788572578.610053:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:37369:x1875453360689536:12345-192.168.202.40@tcp:55:lfs.0 00010000:00010000:3.0:1788572578.610124:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.610142:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 81 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.610163:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 282 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.610172:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 5/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.610210:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 282 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.610224:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 282 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.610244:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382409856:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572578.610410:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 81 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.610420:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 81 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.610429:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382409984:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572578.610452:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102180 going to sleep for 11 seconds 00000100:00100000:2.0:1788572578.618270:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3303:1875453382410240:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618359:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572578.618419:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572578.618426:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3303:1875453382410112:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618460:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572578.618497:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3305:1875453382410496:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618527:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572578.618545:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572578.618550:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3305:1875453382410368:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618573:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572578.618611:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382410112 00000100:00100000:2.0:1788572578.618618:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382410112 00000100:00100000:2.0:1788572578.618668:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 692 00000100:00100000:2.0:1788572578.618677:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0efc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3303:x1875453382410112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618728:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0efc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3303:x1875453382410112:12345-0@lo:400:kworker.0 Request processed in 53us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.618735:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 692 00000100:00100000:2.0:1788572578.618772:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382410240 00000100:00100000:2.0:1788572578.618775:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382410240 00000100:00100000:2.0:1788572578.618798:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382410496 00000100:00100000:2.0:1788572578.618801:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382410496 00000100:00100000:2.0:1788572578.618825:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382410368 00000100:00100000:2.0:1788572578.618828:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382410368 00000100:00100000:2.0:1788572578.618843:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:2.0:1788572578.618849:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1875453382410240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618881:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1875453382410240:12345-0@lo:400:kworker.0 Request processed in 32us (522us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.618887:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00100000:2.0:1788572578.618898:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:2.0:1788572578.618903:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1875453382410496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618934:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1875453382410496:12345-0@lo:400:kworker.0 Request processed in 31us (407us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.618940:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:2.0:1788572578.618951:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1788572578.618956:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1875453382410368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.618982:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1875453382410368:12345-0@lo:400:kworker.0 Request processed in 26us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.618988:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00080000:2.0:1788572578.619025:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572578.619062:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3305:1875453382410368:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.619073:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3305:1875453382410496:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.619103:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3303:1875453382410112:0@lo:400:kworker.0 00000100:00100000:2.0:1788572578.619112:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3303:1875453382410240:0@lo:400:kworker.0 00000100:00100000:0.0:1788572578.632614:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.632759:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382409856:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572578.632775:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382409984:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572578.632802:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.632951:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.640515:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360689664 00000100:00100000:1.0:1788572578.640532:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360689664 00010000:00010000:1.0:1788572578.640588:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 283 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.640632:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360689792 00000100:00100000:1.0:1788572578.640637:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360689792 00010000:00010000:1.0:1788572578.640653:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 82 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.640679:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1177 00000100:00100000:1.0:1788572578.640689:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:2350:x1875453360689664:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.640705:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.640713:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 283 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640731:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 283 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640753:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1663d4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f198f expref: 283 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640770:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572578.640775:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640783:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640790:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572578.640845:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:2350:x1875453360689664:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 157us (8241us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.640852:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1177 00000100:00100000:1.0:1788572578.640886:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1178 00000100:00100000:1.0:1788572578.640893:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:2350:x1875453360689792:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.640906:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.640911:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 82 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640920:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 82 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640937:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb16639c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1957 expref: 82 pid: 5838 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640946:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572578.640950:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640957:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640964:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.640988:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572578.641041:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:2350:x1875453360689792:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 148us (8090us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.641047:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1178 00010000:00010000:1.0:1788572578.641217:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641225:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641460:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641469:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641476:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641482:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.641487:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641494:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641504:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572578.641507:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572578.641511:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.641517:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb1663e2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.641571:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:37369:x1875453360689536:12345-192.168.202.40@tcp:55:lfs.0 Request processed in 31520us (31892us total) trans 4294971080 rc 0/0 00000100:00100000:1.0:1788572578.641581:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4990 00000100:00100000:0.0:1788572578.644244:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.644330:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690048 00000100:00100000:1.0:1788572578.644339:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690048 00000100:00100000:1.0:1788572578.644403:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1086 00000100:00100000:1.0:1788572578.644415:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+282:37369:x1875453360690048:12345-192.168.202.40@tcp:35:lfs.0 00000100:00100000:1.0:1788572578.644655:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:37369:x1875453360690048:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 241us (418us total) trans 4294971081 rc 0/0 00000100:00100000:1.0:1788572578.644663:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1086 00000100:00100000:1.0:1788572578.708135:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.708229:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690176 00000100:00100000:2.0:1788572578.708237:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690176 00000100:00100000:2.0:1788572578.708344:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4991 00000100:00100000:2.0:1788572578.708354:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+283:37370:x1875453360690176:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572578.708376:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572578.708406:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1663e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1996 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708485:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708502:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708518:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708533:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708539:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708544:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708653:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708661:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708669:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572578.708705:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00000004:00200000:2.0:1788572578.708710:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572578.708729:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708754:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708767:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708778:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708785:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708791:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aec00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.708795:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708802:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708810:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572578.708812:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.708816:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708820:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1663f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708830:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708834:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708838:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708841:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.708843:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708848:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708854:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627e3306a00) 00010000:00010000:2.0:1788572578.708857:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.708860:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708865:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1663f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708896:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb1663e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 285 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708910:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 284 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.708918:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627e3306300) 00010000:00010000:2.0:1788572578.708920:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572578.708922:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aea00, rc 0) 00000100:00100000:2.0:1788572578.708979:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+284:37370:x1875453360690176:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 626us (853us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.708986:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4991 00000100:00100000:2.0:1788572578.713870:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.714125:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690304 00000100:00100000:3.0:1788572578.714133:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690304 00000100:00100000:3.0:1788572578.714190:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1461 00000100:00100000:3.0:1788572578.714201:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2348:x1875453360690304:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:3.0:1788572578.714221:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572578.714240:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f199d expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572578.714264:0:13272:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x60:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1788572578.714428:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=926s v=(0 0) 00010000:00010000:3.0:1788572578.714434:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572578.714444:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 75 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.714465:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 75 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.714479:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5200, rc 0) 00000100:00100000:3.0:1788572578.714578:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2348:x1875453360690304:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 379us (715us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572578.714697:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1461 00000100:00100000:1.0:1788572578.714967:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.715568:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690432 00000100:00100000:3.0:1788572578.715576:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690432 00000100:00100000:0.0:1788572578.716912:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 421 00000100:00100000:0.0:1788572578.716931:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+284:2350:x1875453360690432:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572578.716972:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572578.716994:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16640c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19a4 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717069:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717084:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717094:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717111:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717118:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717125:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717190:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717197:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717204:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717210:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.717214:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717221:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717232:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.717236:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.717241:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717247:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166413 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717260:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717265:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717270:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717275:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.717278:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717284:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717290:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xb3dc5f41 (ffff9627c5160300) 00010000:00010000:0.0:1788572578.717292:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.717295:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717301:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16641a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717329:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16640c lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19a4 expref: 285 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717769:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.717771:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572578.717777:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16640c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19a4 expref: 285 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.717790:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1400, rc -2) 00000100:00100000:0.0:1788572578.717872:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+284:2350:x1875453360690432:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 947us (2905us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572578.717880:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 421 00000100:00100000:1.0:1788572578.721064:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.721188:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690560 00000100:00100000:2.0:1788572578.721198:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690560 00000100:00100000:2.0:1788572578.721253:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4992 00000100:00100000:2.0:1788572578.721264:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+284:37370:x1875453360690560:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572578.721400:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572578.721423:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb166421 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19d5 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.721493:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.721507:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.721519:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572578.721553:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572578.721557:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572578.721574:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.721680:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb166421 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 286 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.721818:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 285 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.721826:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572578.721829:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572578.721832:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5aec00, rc 0) 00000100:00100000:2.0:1788572578.721978:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:37370:x1875453360690560:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 715us (923us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572578.721989:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4992 00000100:00100000:2.0:1788572578.728267:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.729679:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690816 00000100:00100000:1.0:1788572578.729693:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690816 00000100:00100000:1.0:1788572578.729776:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4993 00000100:00100000:1.0:1788572578.729785:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+285:37370:x1875453360690816:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:1.0:1788572578.729806:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.729826:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16642f lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19dc expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.729959:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb166436 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.729974:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb166436 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.729986:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb166436 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572578.729998:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00010000:00010000:1.0:1788572578.730148:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16642f lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19dc expref: 288 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.730165:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb166436 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19dc expref: 287 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.730172:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627e3306300) 00010000:00010000:1.0:1788572578.730175:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1788572578.730179:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646a00, rc 0) 00000100:00100000:1.0:1788572578.730243:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:37370:x1875453360690816:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 459us (1989us total) trans 4294971082 rc 0/0 00000100:00100000:1.0:1788572578.730251:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4993 00000100:00100000:0.0:1788572578.734109:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.734304:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360690944 00000100:00100000:3.0:1788572578.734314:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360690944 00000100:00100000:3.0:1788572578.734382:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1462 00000100:00100000:3.0:1788572578.734404:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+75:2347:x1875453360690944:12345-192.168.202.40@tcp:1:lfs.0 00010000:00010000:3.0:1788572578.734452:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.734475:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 75 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.734485:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572578.734489:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572578.734495:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.734539:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 75 pid: 13272 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.734554:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 75 pid: 13272 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.734570:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382410624:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572578.734682:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102180 going to sleep for 11 seconds 00000100:00100000:2.0:1788572578.737835:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.738095:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360691072 00000100:00100000:0.0:1788572578.738104:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360691072 00010000:00010000:0.0:1788572578.738144:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 76 pid: 13272 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.738189:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1179 00000100:00100000:0.0:1788572578.738198:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2349:x1875453360691072:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572578.738214:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572578.738241:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 76 pid: 13272 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.738255:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 76 pid: 13272 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.738292:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572578.738297:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572578.738303:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.738318:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.738350:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572578.738437:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+76:2349:x1875453360691072:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 240us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.738445:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1179 00010000:00010000:3.0:1788572578.739968:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083ae6b3b/0x85ace67feb166405 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f199d expref: 75 pid: 13272 timeout: 1026 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.739995:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382410624:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572578.740045:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740056:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740062:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740092:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740107:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740116:0:20911:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740122:0:20911:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572578.740127:0:20911:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740136:0:20911:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740158:0:20911:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.740166:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166444 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.740246:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2347:x1875453360690944:12345-192.168.202.40@tcp:1:lfs.0 Request processed in 5854us (6148us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572578.740256:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1462 00000100:00100000:2.0:1788572578.748797:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.748897:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360691328 00000100:00100000:3.0:1788572578.748908:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360691328 00000100:00100000:3.0:1788572578.748972:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1087 00000100:00100000:3.0:1788572578.748983:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+287:37370:x1875453360691328:12345-192.168.202.40@tcp:35:lfs.0 00000004:00010000:3.0:1788572578.749090:0:14530:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0xb3:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb166436 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19dc expref: 287 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572578.749367:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971083 00000001:00080000:3.0:1788572578.749830:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971084 00010000:00010000:3.0:1788572578.749899:0:14530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.749915:0:14530:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 290 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.749925:0:14530:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.749961:0:14530:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 290 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.749976:0:14530:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 290 pid: 23058 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.749990:0:14530:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:14530:1875453382410752:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572578.750165:0:14530:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13eb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572578.755542:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.755652:0:14530:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:14530:1875453382410752:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572578.755680:0:14530:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.755797:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360691456 00000100:00100000:1.0:1788572578.755805:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360691456 00010000:00010000:1.0:1788572578.755842:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 291 pid: 23058 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.755897:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1180 00000100:00100000:1.0:1788572578.755906:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+291:2349:x1875453360691456:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572578.755921:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572578.755928:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 291 pid: 23058 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.755940:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 291 pid: 23058 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.755962:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1663fe lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1996 expref: 291 pid: 23058 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.755977:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627e3306300) 00010000:00010000:1.0:1788572578.755983:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.755992:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.756000:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.756073:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572578.756118:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+290:2349:x1875453360691456:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 214us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.756126:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1180 00010000:00010000:3.0:1788572578.756314:0:14530:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.756323:0:14530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572578.756458:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971085 00000001:00080000:3.0:1788572578.756676:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971086 00000001:00080000:3.0:1788572578.756732:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971087 00000100:00100000:3.0:1788572578.756814:0:14530:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:14530:1875453382410880:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:3.0:1788572578.756858:0:14530:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572578.756902:0:14530:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572578.757164:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382410880 00000100:00100000:1.0:1788572578.757170:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382410880 00000100:00100000:1.0:1788572578.757203:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1788572578.757211:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:14530:x1875453382410880:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:1.0:1788572578.757287:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:14530:x1875453382410880:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 78us (430us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572578.757294:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1788572578.757331:0:14530:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:14530:1875453382410880:0@lo:1000:mdt_rdpg00_002.0 00000001:00080000:0.0:1788572578.757381:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971088 00010000:00010000:0.0:1788572578.757430:0:14530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.757440:0:14530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.757446:0:14530:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.757452:0:14530:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.757457:0:14530:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.757473:0:14530:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.757486:0:14530:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627e3306300) 00010000:00010000:0.0:1788572578.757489:0:14530:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1788572578.757493:0:14530:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.757500:0:14530:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb16644b lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 14530 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1788572578.757712:0:14530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0xd4:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1788572578.757796:0:14530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].83, 64 off14896 00000001:00080000:0.0:1788572578.757807:0:14530:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971089 00010000:00010000:0.0:1788572578.757835:0:14530:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627e3306300) 00010000:00010000:0.0:1788572578.757838:0:14530:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1788572578.757845:0:14530:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb166436 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19dc expref: 294 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.757929:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:37370:x1875453360691328:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 8947us (9143us total) trans 4294971090 rc 0/0 00000100:00100000:0.0:1788572578.757937:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1087 00000020:00080000:0.0:1788572578.757941:0:14530:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1788572578.814136:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.814313:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360691584 00000100:00100000:0.0:1788572578.814325:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360691584 00000100:00100000:0.0:1788572578.814398:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4994 00000100:00100000:0.0:1788572578.814414:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:37376:x1875453360691584:12345-192.168.202.40@tcp:101:mkdir.0 00010000:00010000:0.0:1788572578.814439:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572578.814469:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166452 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19e3 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.814544:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166459 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.814562:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166459 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.814575:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166459 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572578.814613:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:0.0:1788572578.814617:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572578.814638:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166459 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.814667:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166452 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19e3 expref: 83 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.814686:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166459 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19e3 expref: 82 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.814694:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.814698:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572578.814702:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0c00, rc 0) 00000100:00100000:0.0:1788572578.814964:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:37376:x1875453360691584:12345-192.168.202.40@tcp:101:mkdir.0 Request processed in 555us (836us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.814973:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4994 00000100:00100000:1.0:1788572578.831942:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.832782:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360691712 00000100:00100000:0.0:1788572578.832796:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360691712 00000100:00100000:0.0:1788572578.832882:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 423 00000100:00100000:0.0:1788572578.832895:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:2348:x1875453360691712:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572578.832938:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572578.832958:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166460 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19ea expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833258:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833274:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833286:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833301:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833308:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833315:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833370:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833378:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833385:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833401:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.833405:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833416:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833429:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.833432:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.833437:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833443:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166467 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833456:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833462:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833467:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833471:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572578.833473:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833479:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833485:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x79e6baae (ffff9627c5160300) 00010000:00010000:0.0:1788572578.833487:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572578.833490:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833707:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16646e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833744:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166460 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ea expref: 83 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833755:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.833757:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572578.833760:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166460 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ea expref: 83 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.833770:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1400, rc -2) 00000100:00100000:0.0:1788572578.834047:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:2348:x1875453360691712:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1154us (2114us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572578.834056:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 423 00000100:00100000:2.0:1788572578.838742:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.840465:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360691968 00000100:00100000:0.0:1788572578.840478:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360691968 00000100:00100000:0.0:1788572578.840592:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4995 00000100:00100000:0.0:1788572578.840603:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:37377:x1875453360691968:12345-192.168.202.40@tcp:36:rm.0 00010000:00010000:0.0:1788572578.840660:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788572578.840687:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb165f82 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17f2 expref: 82 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.840706:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572578.840716:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166459 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19e3 expref: 81 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.840731:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572578.840734:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572578.840758:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.840772:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 292 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.840780:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.840814:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 292 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.840828:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 292 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.840842:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4bbbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382411008:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572578.840966:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32600 going to sleep for 11 seconds 00000100:00100000:1.0:1788572578.844427:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.844547:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4bbbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382411008:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572578.844578:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.844648:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360692096 00000100:00100000:3.0:1788572578.844656:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360692096 00010000:00010000:3.0:1788572578.844695:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 293 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.844740:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1181 00000100:00100000:3.0:1788572578.844749:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:2350:x1875453360692096:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572578.844767:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572578.844774:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 293 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.844787:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 293 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.844813:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb166428 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19d5 expref: 293 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.844828:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572578.844833:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.844844:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.844871:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.844902:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572578.844960:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+292:2350:x1875453360692096:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 211us (542us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572578.844967:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1181 00010000:00010000:2.0:1788572578.845254:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845271:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845297:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845306:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845314:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845437:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845447:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 291 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845454:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845489:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 291 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.845503:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 291 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.845518:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382411136:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572578.845573:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572578.850478:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.850557:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360692224 00000100:00100000:3.0:1788572578.850567:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360692224 00010000:00010000:3.0:1788572578.850609:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 292 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.850652:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1182 00000100:00100000:3.0:1788572578.850662:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+292:2349:x1875453360692224:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572578.850677:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572578.850684:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 292 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.850696:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 292 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.850719:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572578.850725:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.850735:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.850743:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.850763:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572578.850824:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+292:2349:x1875453360692224:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 163us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572578.850831:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1182 00010000:00010000:2.0:1788572578.851755:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb165ff2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xaf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f181c expref: 291 pid: 5839 timeout: 1026 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.851776:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382411136:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.851798:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.851807:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.851813:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1788572578.852499:0:5837:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].84, 64 off14960 00010000:00010000:2.0:1788572578.852571:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 3/0,1 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852585:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852598:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852605:0:5837:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852612:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852618:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852823:0:5837:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788572578.852847:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852854:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852859:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852865:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5afc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.852870:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852878:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852890:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:2.0:1788572578.852893:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788572578.852899:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852905:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166483 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852922:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852928:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852934:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852940:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.852942:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852949:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852957:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572578.852959:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.852962:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.852968:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166475 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572578.852980:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000acad4537 reply state 000000001dc7a5bb transno 4294971091 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.852993:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572578.853108:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+82:37377:x1875453360691968:12345-192.168.202.40@tcp:36:rm.0 Request processed in 12505us (14375us total) trans 4294971091 rc 0/0 00000100:00100000:2.0:1788572578.853117:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4995 00010000:00010000:1.0:1788572578.858807:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572578.858823:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.858834:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572578.858839:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.858869:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.858890:0:5825:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xaf:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:1.0:1788572578.858893:0:5825:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572578.858899:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 1/0,0 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.858906:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb16648a lrc: 0/0,0 mode: --/PW res: [0x200000402:0xaf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.859246:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572578.859250:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572578.859252:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572578.859540:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.859551:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.859557:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.859562:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.859566:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.859573:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.859583:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627e3306a00) 00010000:00010000:3.0:1788572578.859586:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.859590:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.859599:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb16647c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.880970:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572578.881319:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360692480 00000100:00100000:1.0:1788572578.881332:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360692480 00000100:00100000:1.0:1788572578.881406:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4996 00000100:00100000:1.0:1788572578.881417:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+81:37378:x1875453360692480:12345-192.168.202.40@tcp:101:dd.0 00010000:00010000:1.0:1788572578.881440:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572578.881461:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166491 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19f1 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.881597:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.881614:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.881627:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.881642:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.881649:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.881655:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1788572578.882236:0:5838:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x60:0x0], index 1: 0 00010000:00010000:1.0:1788572578.882416:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882426:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882435:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882480:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882487:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882492:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882498:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.882502:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882510:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882523:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572578.882526:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.882531:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882542:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166498 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572578.882554:0:5838:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a8995b7a reply state 0000000075006a53 transno 4294971092 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572578.882563:0:5838:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1788572578.882595:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166491 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 84 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882609:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.882616:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:1.0:1788572578.882618:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572578.882621:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646000, rc 0) 00000100:00100000:1.0:1788572578.882704:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:37378:x1875453360692480:12345-192.168.202.40@tcp:101:dd.0 Request processed in 1287us (1743us total) trans 4294971092 rc 0/0 00000100:00100000:1.0:1788572578.882713:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4996 00010000:00010000:1.0:1788572578.885774:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.885792:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.885799:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.885804:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572578.885810:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.885818:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.885836:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627ff5abc00) 00010000:00010000:1.0:1788572578.885839:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572578.885844:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572578.885852:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16649f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.887071:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.887191:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360692736 00000100:00100000:2.0:1788572578.887202:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360692736 00000100:00100000:2.0:1788572578.887271:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1088 00000100:00100000:2.0:1788572578.887283:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13c050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:37378:x1875453360692736:12345-192.168.202.40@tcp:35:dd.0 00000100:00100000:2.0:1788572578.887817:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13c050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:37378:x1875453360692736:12345-192.168.202.40@tcp:35:dd.0 Request processed in 535us (752us total) trans 4294971093 rc 0/0 00000100:00100000:2.0:1788572578.887826:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1088 00010000:00010000:3.0:1788572578.939457:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572578.940717:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.940821:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360692864 00000100:00100000:3.0:1788572578.940831:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360692864 00000100:00100000:3.0:1788572578.940956:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 424 00000100:00100000:3.0:1788572578.940967:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+291:2347:x1875453360692864:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572578.941042:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572578.941062:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1664b4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19f8 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.941123:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942472:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942493:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942515:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942524:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942530:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942828:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942836:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.942843:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943116:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.943121:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943130:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943218:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:3.0:1788572578.943221:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.943227:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943233:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664bb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943245:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943249:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943256:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943262:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.943264:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943269:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943277:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x8799b2a3 (ffff9626c4b10d00) 00010000:00010000:3.0:1788572578.943279:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.943282:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943287:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x97:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943329:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1664b4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f8 expref: 292 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943339:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x97:0x0].0x0 (ffff9627c2becf00) 00010000:00010000:3.0:1788572578.943341:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572578.943345:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb1664b4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x97:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f8 expref: 292 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.943354:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5800, rc -2) 00000100:00100000:3.0:1788572578.943497:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4bb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+291:2347:x1875453360692864:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 2531us (2789us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572578.943507:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 424 00000100:00100000:3.0:1788572578.950870:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.950942:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360693120 00000100:00100000:2.0:1788572578.950953:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360693120 00000100:00100000:2.0:1788572578.951056:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4997 00000100:00100000:2.0:1788572578.951067:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+291:37381:x1875453360693120:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572578.951090:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572578.951115:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664c9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f19ff expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951211:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951228:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951239:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951254:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951261:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951268:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951400:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951409:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951417:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572578.951428:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00010000:00010000:2.0:1788572578.951534:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951541:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951548:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951554:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.951558:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951571:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951583:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572578.951586:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.951592:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951598:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb1664d0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951610:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951617:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951623:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951629:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.951632:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951638:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951646:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627e3306400) 00010000:00010000:2.0:1788572578.951648:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572578.951651:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951658:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1664d7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951700:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664c9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 294 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951715:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 293 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.951723:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:2.0:1788572578.951725:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572578.951728:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af400, rc 0) 00000100:00100000:2.0:1788572578.951803:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:37381:x1875453360693120:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 738us (955us total) trans 4294971094 rc 0/0 00000100:00100000:2.0:1788572578.951811:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4997 00000100:00100000:0.0:1788572578.958447:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572578.958534:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360693376 00000100:00100000:2.0:1788572578.958545:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360693376 00000100:00100000:2.0:1788572578.958613:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4998 00000100:00100000:2.0:1788572578.958625:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:37381:x1875453360693376:12345-192.168.202.40@tcp:55:lfs.0 00010000:00010000:2.0:1788572578.958700:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.958718:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.958726:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 293 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.958739:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 5/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.958777:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 293 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.958793:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 293 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.958810:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382411264:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.958943:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 83 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.958953:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 83 pid: 5838 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.958963:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382411392:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572578.958987:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17dcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572578.963579:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.963689:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360693504 00000100:00100000:3.0:1788572578.963698:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360693504 00010000:00010000:3.0:1788572578.963739:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 84 pid: 5838 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.963781:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1183 00000100:00100000:3.0:1788572578.963789:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:2350:x1875453360693504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572578.963803:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572578.963810:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 84 pid: 5838 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.963822:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 84 pid: 5838 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.963858:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:3.0:1788572578.963864:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.963873:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.963878:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572578.963928:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:2350:x1875453360693504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 139us (357us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572578.963934:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1183 00000100:00100000:2.0:1788572578.965201:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2398000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382411264:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.965218:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb1664a6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19f1 expref: 83 pid: 5838 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.965234:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382411392:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572578.965256:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572578.968108:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.968274:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360693632 00000100:00100000:0.0:1788572578.968285:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360693632 00010000:00010000:0.0:1788572578.968329:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 294 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572578.968374:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1184 00000100:00100000:0.0:1788572578.968394:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+294:2350:x1875453360693632:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572578.968411:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572578.968419:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 294 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.968432:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 294 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.968453:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb1664de lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f19ff expref: 294 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.968469:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:0.0:1788572578.968474:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.968484:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.968491:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572578.968518:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572578.968576:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:2350:x1875453360693632:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 193us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572578.968583:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1184 00010000:00010000:2.0:1788572578.969528:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969539:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969767:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969775:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969782:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969787:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5afc00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572578.969792:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969799:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969812:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:2.0:1788572578.969816:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572578.969821:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572578.969827:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb1664ec lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572578.970288:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4b32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:37381:x1875453360693376:12345-192.168.202.40@tcp:55:lfs.0 Request processed in 11662us (11846us total) trans 4294971095 rc 0/0 00000100:00100000:2.0:1788572578.970300:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4998 00000100:00100000:3.0:1788572578.973671:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.973740:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360693888 00000100:00100000:0.0:1788572578.973750:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360693888 00000100:00100000:0.0:1788572578.973806:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1089 00000100:00100000:0.0:1788572578.973817:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+293:37381:x1875453360693888:12345-192.168.202.40@tcp:35:lfs.0 00000100:00100000:0.0:1788572578.974149:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+294:37381:x1875453360693888:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 320us (474us total) trans 4294971096 rc 0/0 00000100:00100000:0.0:1788572578.974157:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1089 00000100:00100000:1.0:1788572578.991722:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572578.991861:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694016 00000100:00100000:3.0:1788572578.991873:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694016 00000100:00100000:3.0:1788572578.991938:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 4999 00000100:00100000:3.0:1788572578.991948:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+294:37382:x1875453360694016:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:3.0:1788572578.991976:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572578.991997:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664f3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a06 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992114:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992129:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992407:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992425:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992433:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992439:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992531:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992540:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992547:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572578.992582:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00000004:00200000:3.0:1788572578.992587:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572578.992607:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992618:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992626:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992633:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992639:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992645:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.992650:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992658:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992670:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572578.992673:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.992678:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992684:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1664fa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992702:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992709:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992715:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992720:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572578.992723:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992729:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992735:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9626c4b10d00) 00010000:00010000:3.0:1788572578.992738:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572578.992741:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992747:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166501 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992779:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1664f3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 296 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992792:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 295 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572578.992803:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10300) 00010000:00010000:3.0:1788572578.992805:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572578.992807:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5000, rc 0) 00000100:00100000:3.0:1788572578.993080:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+295:37382:x1875453360694016:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 1132us (1370us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572578.993089:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 4999 00000100:00100000:2.0:1788572578.996653:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572578.999869:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694144 00000100:00100000:0.0:1788572578.999885:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694144 00000100:00100000:0.0:1788572578.999960:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 425 00000100:00100000:0.0:1788572578.999971:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+295:2350:x1875453360694144:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572579.000030:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572579.000053:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16650f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a0d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000121:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000150:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000163:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000183:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000190:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000196:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000258:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000267:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000274:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000280:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572579.000284:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000292:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000304:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572579.000307:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572579.000311:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000318:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166516 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000333:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000339:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000344:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000348:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572579.000351:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000356:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000363:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x79e6baae (ffff9627c5160300) 00010000:00010000:0.0:1788572579.000365:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572579.000368:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000374:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16651d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x79e6baae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000419:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16650f lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a0d expref: 296 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000430:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572579.000432:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572579.000437:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb16650f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a0d expref: 296 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.000447:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1c00, rc -2) 00000100:00100000:0.0:1788572579.000522:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+295:2350:x1875453360694144:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 552us (3880us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572579.000530:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 425 00000100:00100000:1.0:1788572579.003574:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.004256:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694272 00000100:00100000:0.0:1788572579.004267:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694272 00000100:00100000:0.0:1788572579.004328:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1463 00000100:00100000:0.0:1788572579.004339:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2348:x1875453360694272:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572579.004356:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572579.004378:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a14 expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572579.004415:0:6681:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x60:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572579.004433:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=927s v=(0 0) 00010000:00010000:0.0:1788572579.004437:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572579.004446:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 86 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.004463:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 86 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.004475:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0a00, rc 0) 00000100:00100000:0.0:1788572579.004526:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2348:x1875453360694272:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 190us (963us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572579.004536:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1463 00000100:00100000:1.0:1788572579.008106:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.008520:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694400 00000100:00100000:2.0:1788572579.008532:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694400 00000100:00100000:2.0:1788572579.008645:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5000 00000100:00100000:2.0:1788572579.008657:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+295:37382:x1875453360694400:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:2.0:1788572579.008677:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.008698:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16652b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a1b expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.008769:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.008784:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.008795:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572579.008829:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd0:0x0]: returning size 16384 00000004:00200000:2.0:1788572579.008833:0:23058:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572579.009106:0:23058:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.009132:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb16652b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 297 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.009195:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 296 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.009205:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572579.009207:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572579.009210:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af800, rc 0) 00000100:00100000:2.0:1788572579.009361:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7bc7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+296:37382:x1875453360694400:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 705us (1260us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.009369:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5000 00000100:00080000:0.0:1788572579.009446:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572579.014230:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.014333:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694656 00000100:00100000:0.0:1788572579.014345:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694656 00000100:00100000:0.0:1788572579.014445:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5001 00000100:00100000:0.0:1788572579.014456:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+296:37382:x1875453360694656:12345-192.168.202.40@tcp:101:lfs.0 00010000:00010000:0.0:1788572579.014478:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572579.014500:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166539 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a22 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.014605:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166540 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.014620:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166540 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.014631:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166540 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572579.014644:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00010000:00010000:0.0:1788572579.014774:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166539 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a22 expref: 299 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.014791:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166540 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a22 expref: 298 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.014799:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10300) 00010000:00010000:0.0:1788572579.014802:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1788572579.014807:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0200, rc 0) 00000100:00100000:0.0:1788572579.015104:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+298:37382:x1875453360694656:12345-192.168.202.40@tcp:101:lfs.0 Request processed in 648us (879us total) trans 4294971097 rc 0/0 00000100:00100000:0.0:1788572579.015112:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5001 00000100:00100000:2.0:1788572579.018816:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.019288:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694784 00000100:00100000:0.0:1788572579.019298:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694784 00000100:00100000:0.0:1788572579.019400:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1464 00000100:00100000:0.0:1788572579.019412:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2350:x1875453360694784:12345-192.168.202.40@tcp:1:lfs.0 00010000:00010000:0.0:1788572579.019460:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.019479:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 86 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.019487:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572579.019492:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572579.019497:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 4/0,1 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.019538:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 86 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.019553:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 86 pid: 6681 timeout: 1027 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.019569:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382411520:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572579.019698:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32600 going to sleep for 11 seconds 00000100:00100000:2.0:1788572579.023491:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.023600:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382411520:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572579.023626:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572579.023674:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360694912 00000100:00100000:1.0:1788572579.023682:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360694912 00010000:00010000:1.0:1788572579.023722:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 87 pid: 6681 timeout: 1027 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572579.023802:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1185 00000100:00100000:1.0:1788572579.023813:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+87:2349:x1875453360694912:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572579.023827:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572579.023864:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 87 pid: 6681 timeout: 1027 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.023882:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 87 pid: 6681 timeout: 1027 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.023905:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4462e30/0x85ace67feb166524 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x60:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a14 expref: 87 pid: 6681 timeout: 1027 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.023921:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572579.023926:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572579.023931:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.023941:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.024001:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572579.024078:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2349:x1875453360694912:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 266us (596us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572579.024086:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1185 00010000:00010000:0.0:1788572579.025113:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025128:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025166:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025176:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025185:0:6679:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025192:0:6679:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572579.025197:0:6679:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025206:0:6679:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025228:0:6679:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.025236:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb16654e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.025312:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2398000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2350:x1875453360694784:12345-192.168.202.40@tcp:1:lfs.0 Request processed in 5902us (6504us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572579.025321:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1464 00000100:00100000:2.0:1788572579.030313:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.030400:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360695168 00000100:00100000:0.0:1788572579.030411:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360695168 00000100:00100000:0.0:1788572579.030471:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1090 00000100:00100000:0.0:1788572579.030482:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+298:37382:x1875453360695168:12345-192.168.202.40@tcp:35:lfs.0 00000004:00010000:0.0:1788572579.030548:0:5840:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0xb4:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166540 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a22 expref: 298 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572579.030782:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971098 00000001:00080000:0.0:1788572579.031171:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971099 00010000:00010000:0.0:1788572579.031224:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.031239:0:5840:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 301 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.031249:0:5840:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.031288:0:5840:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 301 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.031303:0:5840:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 301 pid: 9040 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.031317:0:5840:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1d8aa00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5840:1875453382411648:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572579.031411:0:5840:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788572579.035231:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.035339:0:5840:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1d8aa00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5840:1875453382411648:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572579.035370:0:5840:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572579.035458:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360695296 00000100:00100000:3.0:1788572579.035468:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360695296 00010000:00010000:3.0:1788572579.035515:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 302 pid: 9040 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572579.035558:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1186 00000100:00100000:3.0:1788572579.035569:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+302:2349:x1875453360695296:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572579.035586:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572579.035594:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 302 pid: 9040 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.035605:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 302 pid: 9040 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.035626:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166508 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a06 expref: 302 pid: 9040 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.035641:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10300) 00010000:00010000:3.0:1788572579.035646:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.035656:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.035664:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.035691:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572579.035748:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+301:2349:x1875453360695296:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 180us (526us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572579.035755:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1186 00010000:00010000:2.0:1788572579.036595:0:5840:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.036605:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788572579.036758:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971100 00000001:00080000:2.0:1788572579.037000:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971101 00000001:00080000:2.0:1788572579.037208:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971102 00000100:00100000:2.0:1788572579.037295:0:5840:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239a680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5840:1875453382411776:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1788572579.037354:0:5840:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572579.037436:0:5840:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788572579.038473:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382411776 00000100:00100000:3.0:1788572579.038484:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382411776 00000100:00100000:3.0:1788572579.038601:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:3.0:1788572579.038611:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5840:x1875453382411776:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1788572579.038815:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5840:x1875453382411776:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 205us (1461us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572579.038823:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00080000:0.0:1788572579.039116:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788572579.039249:0:5840:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239a680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5840:1875453382411776:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1788572579.039330:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971103 00010000:00010000:2.0:1788572579.039392:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.039410:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.039418:0:5840:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.039424:0:5840:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572579.039430:0:5840:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.039438:0:5840:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.039456:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10300) 00010000:00010000:2.0:1788572579.039459:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1788572579.039465:0:5840:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.039472:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166555 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572579.039705:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0xd5:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1788572579.039781:0:5840:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].81, 64 off14768 00000001:00080000:2.0:1788572579.039793:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971104 00010000:00010000:2.0:1788572579.039827:0:5840:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10300) 00010000:00010000:2.0:1788572579.039830:0:5840:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1788572579.039838:0:5840:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166540 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a22 expref: 305 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572579.040411:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+304:37382:x1875453360695168:12345-192.168.202.40@tcp:35:lfs.0 Request processed in 9901us (10081us total) trans 4294971105 rc 0/0 00000100:00100000:2.0:1788572579.040423:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1090 00000020:00080000:2.0:1788572579.040428:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1788572579.130524:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572579.130547:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4920e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382411904:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788572579.130624:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572579.130805:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382411904 00000100:00100000:0.0:1788572579.130814:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382411904 00000100:00100000:0.0:1788572579.131045:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:0.0:1788572579.131056:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382411904:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788572579.131094:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 198901760 pending 0 free 3853516800 avail 3851419648 00000100:00100000:0.0:1788572579.131255:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382411904:12345-0@lo:13:osp-pre-1-0.0 Request processed in 200us (629us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572579.131263:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:2.0:1788572579.131359:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4920e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382411904:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788572579.404049:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.404145:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360695552 00000100:00100000:1.0:1788572579.404155:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360695552 00000100:00100000:1.0:1788572579.404218:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5002 00000100:00100000:1.0:1788572579.404233:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+304:37407:x1875453360695552:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:1.0:1788572579.404258:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572579.404322:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16655c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a29 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404466:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404481:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404494:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404509:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404527:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404534:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404661:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb166571 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404670:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb166571 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404677:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb166571 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572579.404685:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb0:0x0]: returning size 0 00010000:00010000:1.0:1788572579.404783:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404793:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404799:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404804:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.404809:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404816:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404828:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572579.404831:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572579.404837:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404843:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb166563 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404855:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404860:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404866:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404870:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.404874:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404880:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404886:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x897cbe23 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572579.404889:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572579.404891:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404898:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb16656a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x897cbe23 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404944:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16655c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a29 expref: 307 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404959:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb166571 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a29 expref: 306 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.404966:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572579.404968:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572579.404970:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646000, rc 0) 00000100:00100000:1.0:1788572579.405093:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+306:37407:x1875453360695552:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 865us (1092us total) trans 4294971106 rc 0/0 00000100:00100000:1.0:1788572579.405100:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5002 00000100:00100000:3.0:1788572579.419235:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.419354:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360695808 00000100:00100000:2.0:1788572579.419374:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360695808 00000100:00100000:3.0:1788572579.419593:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 380 00000100:00100000:3.0:1788572579.419609:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+306:37407:x1875453360695808:12345-192.168.202.40@tcp:103:md5sum.0 00010000:00010000:3.0:1788572579.419628:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572579.419655:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646000/0x85ace67feb166571 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a29 expref: 306 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.419675:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:3.0:1788572579.419687:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572579.419808:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:37407:x1875453360695808:12345-192.168.202.40@tcp:103:md5sum.0 Request processed in 204us (581us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572579.419816:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 380 00000100:00100000:2.0:1788572579.428849:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.429434:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360695936 00000100:00100000:1.0:1788572579.429445:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360695936 00000100:00100000:1.0:1788572579.429521:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5003 00000100:00100000:1.0:1788572579.429533:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:37407:x1875453360695936:12345-192.168.202.40@tcp:58:md5sum.0 00010000:00010000:1.0:1788572579.429648:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.429667:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.429680:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572579.429878:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].174, 136 off31648 00000100:00100000:1.0:1788572579.429978:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:37407:x1875453360695936:12345-192.168.202.40@tcp:58:md5sum.0 Request processed in 446us (1136us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572579.429987:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5003 00000100:00100000:0.0:1788572579.433822:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.439431:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572579.439573:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360696192 00000100:00100000:3.0:1788572579.439585:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360696192 00000100:00100000:3.0:1788572579.439646:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 427 00000100:00100000:3.0:1788572579.439656:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:2348:x1875453360696192:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572579.439695:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572579.439718:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb166586 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a37 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.439784:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.439801:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.439813:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.439827:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.439834:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.439850:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440196:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440210:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440216:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440222:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572579.440227:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440234:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440247:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572579.440250:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572579.440255:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440261:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb16658d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440273:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440280:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440286:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440290:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572579.440294:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440299:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440307:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x8799b2a3 (ffff9626c4b10d00) 00010000:00010000:3.0:1788572579.440309:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572579.440312:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440318:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb166594 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8c:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440349:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb166586 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a37 expref: 306 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440359:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8c:0x0].0x0 (ffff9627d1112500) 00010000:00010000:3.0:1788572579.440362:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572579.440376:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb166586 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x8c:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a37 expref: 306 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.440387:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4000, rc -2) 00000100:00100000:3.0:1788572579.440459:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:2348:x1875453360696192:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 804us (1040us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572579.440468:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 427 00000100:00100000:2.0:1788572579.441692:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360696064 00000100:00100000:2.0:1788572579.441705:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360696064 00000100:00100000:2.0:1788572579.441751:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5004 00000100:00100000:2.0:1788572579.441760:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:37407:x1875453360696064:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572579.441779:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.441798:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a30 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.441968:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 2/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.441980:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 4/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 306 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.442000:0:5839:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.442042:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 306 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.442055:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:2.0:1788572579.442058:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572579.442062:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 306 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.442068:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5af400, rc 0) 00000100:00100000:2.0:1788572579.442126:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+306:37407:x1875453360696064:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 366us (8314us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.442133:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5004 00000040:00080000:2.0:1788572579.770691:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572579.770701:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572579.770953:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 175 count 7 00000040:00080000:2.0:1788572579.770961:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1788572579.771181:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239aa00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382412032:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572579.772034:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572579.777494:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239aa00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382412032:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572579.779913:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.789309:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360696320 00000100:00100000:2.0:1788572579.789325:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360696320 00000100:00100000:2.0:1788572579.789407:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5005 00000100:00100000:2.0:1788572579.789417:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:37410:x1875453360696320:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572579.789454:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb0:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572579.789579:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:37410:x1875453360696320:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 162us (9677us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.789588:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5005 00000100:00100000:0.0:1788572579.798072:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.798397:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360696448 00000100:00100000:1.0:1788572579.798409:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360696448 00000100:00100000:1.0:1788572579.798474:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5006 00000100:00100000:1.0:1788572579.798487:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:37410:x1875453360696448:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1788572579.798567:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798587:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798599:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798615:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798627:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798633:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798658:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798666:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798673:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572579.798718:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb0:0x0]: returning size 0 00000004:00200000:1.0:1788572579.798723:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572579.798738:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798750:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798756:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798763:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798769:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.798773:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798782:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798794:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572579.798798:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572579.798803:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798808:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1665a2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798820:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798840:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798845:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798850:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.798853:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798859:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798866:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xab32668c (ffff9627ff5ab000) 00010000:00010000:1.0:1788572579.798868:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572579.798871:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798881:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb1665a9 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xab32668c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798897:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798903:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798908:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798913:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.798916:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798922:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798929:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572579.798931:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572579.798934:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.798940:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665b0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572579.799224:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:37410:x1875453360696448:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 737us (1162us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572579.799234:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5006 00000100:00100000:0.0:1788572579.809502:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.809607:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360696704 00000100:00100000:2.0:1788572579.809617:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360696704 00000100:00100000:2.0:1788572579.809686:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5007 00000100:00100000:2.0:1788572579.809697:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+83:37410:x1875453360696704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.809718:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.809740:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665b7 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a3e expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.809853:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.809869:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.809880:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572579.809891:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb0:0x0]: returning size 0 00010000:00010000:2.0:1788572579.809988:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.809995:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810041:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810048:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5ae600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572579.810053:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810063:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810077:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:2.0:1788572579.810080:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788572579.810086:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810092:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb1665be lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810147:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665b7 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a3e expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810158:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:2.0:1788572579.810161:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572579.810166:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665b7 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a3e expref: 85 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.810176:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae800, rc 0) 00000100:00100000:2.0:1788572579.810243:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:37410:x1875453360696704:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 548us (752us total) trans 4294971107 rc 301/301 00000100:00100000:2.0:1788572579.810319:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5007 00000100:00100000:3.0:1788572579.816856:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.816934:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360696832 00000100:00100000:2.0:1788572579.816944:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360696832 00000100:00100000:2.0:1788572579.817032:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5008 00000100:00100000:2.0:1788572579.817043:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:37410:x1875453360696832:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572579.817081:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb0:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572579.817196:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:37410:x1875453360696832:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 154us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.817203:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5008 00000100:00100000:0.0:1788572579.827897:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.828381:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360697088 00000100:00100000:2.0:1788572579.828392:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360697088 00000100:00100000:2.0:1788572579.828456:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5009 00000100:00100000:2.0:1788572579.828468:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+84:37410:x1875453360697088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.828492:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.828514:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665cc lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a45 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.829071:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665d3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.829086:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.829098:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665d3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572579.829122:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb5:0x0]: returning size 0 00010000:00010000:2.0:1788572579.829171:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665cc lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a45 expref: 87 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.829190:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665d3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a45 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.829204:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572579.829207:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572579.829211:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae400, rc 0) 00000100:00100000:2.0:1788572579.829700:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e786dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:37410:x1875453360697088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1232us (1809us total) trans 4294971108 rc 0/0 00000100:00100000:2.0:1788572579.829710:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5009 00000100:00100000:0.0:1788572579.834936:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.835707:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360697344 00000100:00100000:2.0:1788572579.835719:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360697344 00000100:00100000:2.0:1788572579.835778:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5010 00000100:00100000:2.0:1788572579.835789:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:37410:x1875453360697344:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.835877:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572579.835904:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665d3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a45 expref: 86 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.835922:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572579.835933:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572579.835956:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.835970:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 305 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.835979:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 4/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.836047:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 305 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.836064:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 305 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572579.836080:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382412160:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572579.836176:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572579.840284:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382412160:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572579.840317:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.841802:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.842037:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360697472 00000100:00100000:1.0:1788572579.842044:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360697472 00010000:00010000:1.0:1788572579.842082:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 306 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572579.842129:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1187 00000100:00100000:1.0:1788572579.842138:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+306:2349:x1875453360697472:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572579.842153:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572579.842160:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 306 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.842172:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 306 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.842192:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af800/0x85ace67feb166532 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a1b expref: 306 pid: 23058 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.842206:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572579.842211:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.842220:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.842227:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.842784:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572579.842913:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+305:2349:x1875453360697472:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 772us (1114us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572579.842925:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1187 00010000:00010000:0.0:1788572579.843075:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843085:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843119:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843131:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843139:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843215:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843221:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843228:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843234:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aec00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572579.843238:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843263:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843275:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572579.843278:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572579.843282:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843288:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aec00/0x85ace67feb1665e1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843299:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843304:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843310:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843315:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572579.843317:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843323:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843330:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x15718704 (ffff9627c5160a00) 00010000:00010000:0.0:1788572579.843333:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572579.843336:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.843345:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1665e8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0x15718704 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.843416:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:37410:x1875453360697344:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 7627us (8488us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572579.843424:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5010 00000100:00100000:3.0:1788572579.872124:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.872225:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360697600 00000100:00100000:2.0:1788572579.872235:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360697600 00000100:00100000:2.0:1788572579.872312:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5011 00000100:00100000:2.0:1788572579.872323:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+85:37410:x1875453360697600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.872344:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.872379:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665ef lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a4c expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.872414:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.872427:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.872439:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.872552:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb1665ef lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 87 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.872571:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 86 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.872585:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572579.872588:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572579.872592:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae800, rc 0) 00000100:00100000:2.0:1788572579.872662:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:37410:x1875453360697600:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 341us (559us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.872669:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5011 00000100:00100000:0.0:1788572579.875957:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.876220:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360697856 00000100:00100000:1.0:1788572579.876231:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360697856 00000100:00100000:1.0:1788572579.876394:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5012 00000100:00100000:1.0:1788572579.876406:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+86:37410:x1875453360697856:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572579.876497:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.876514:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.876525:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572579.876741:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006f1e49f6 reply state 00000000f1fc1357 transno 4294971109 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572579.876751:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572579.881691:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:37410:x1875453360697856:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 5273us (5728us total) trans 4294971109 rc 0/0 00000100:00100000:1.0:1788572579.881706:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5012 00010000:00010000:1.0:1788572579.888192:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.888212:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.888220:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.888231:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.888236:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.888246:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.888278:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:1.0:1788572579.888281:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572579.888287:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.888294:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646a00/0x85ace67feb1665fd lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.890476:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.890560:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360697984 00000100:00100000:2.0:1788572579.890571:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360697984 00000100:00100000:2.0:1788572579.890654:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5013 00000100:00100000:2.0:1788572579.890666:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+87:37410:x1875453360697984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.890692:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.890712:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a53 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.890782:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.890798:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 88 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.890807:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572579.890810:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572579.890813:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae600, rc 0) 00000100:00100000:2.0:1788572579.890889:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c5432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:37410:x1875453360697984:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 225us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.890899:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5013 00000100:00100000:0.0:1788572579.895807:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.895900:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360698112 00000100:00100000:1.0:1788572579.895910:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360698112 00000100:00100000:1.0:1788572579.895972:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1091 00000100:00100000:1.0:1788572579.895982:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:37410:x1875453360698112:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1788572579.896136:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:37410:x1875453360698112:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 Request processed in 155us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572579.896144:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1091 00000100:00100000:1.0:1788572579.899761:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.899868:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360698368 00000100:00100000:2.0:1788572579.899879:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360698368 00000100:00100000:2.0:1788572579.899940:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1092 00000100:00100000:2.0:1788572579.899951:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+88:37410:x1875453360698368:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572579.900486:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:37410:x1875453360698368:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 302us (499us total) trans 4294971110 rc 0/0 00000100:00100000:2.0:1788572579.900496:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1092 00000100:00100000:0.0:1788572579.904219:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.904481:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360698624 00000100:00100000:2.0:1788572579.904491:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360698624 00000100:00100000:2.0:1788572579.904556:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5014 00000100:00100000:2.0:1788572579.904566:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:37410:x1875453360698624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572579.904700:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:37410:x1875453360698624:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 136us (491us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.904708:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5014 00000100:00100000:0.0:1788572579.913654:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.913754:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360698880 00000100:00100000:2.0:1788572579.913764:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360698880 00000100:00100000:2.0:1788572579.913835:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5015 00000100:00100000:2.0:1788572579.913852:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:37410:x1875453360698880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.913874:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.913897:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb16660b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a5a expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.914055:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.914076:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.914086:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.914133:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 90 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.914149:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 90 pid: 5839 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572579.914166:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c239aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382412288:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572579.914282:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572579.919392:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.919536:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c239aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382412288:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572579.919571:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.921997:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360699008 00000100:00100000:0.0:1788572579.922199:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360699008 00010000:00010000:0.0:1788572579.922273:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 91 pid: 5839 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.922324:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1188 00000100:00100000:0.0:1788572579.922335:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:2349:x1875453360699008:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572579.922365:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572579.922373:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 91 pid: 5839 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.922388:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 91 pid: 5839 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.922411:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae600/0x85ace67feb166604 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a53 expref: 91 pid: 5839 timeout: 1027 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.922431:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:0.0:1788572579.922436:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.922448:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.922456:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.922493:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572579.922563:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:2349:x1875453360699008:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 229us (3182us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572579.922570:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1188 00010000:00010000:1.0:1788572579.923852:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.923868:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572579.923891:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb5:0x0]: returning size 0 00000004:00080000:1.0:1788572579.924138:0:23058:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x61:0x0], index 0: 0 00010000:00010000:1.0:1788572579.924329:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924337:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924344:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924359:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572579.924364:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924372:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924387:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:1.0:1788572579.924390:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572579.924396:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924402:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb166612 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924470:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb16660b lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a5a expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924481:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572579.924484:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572579.924488:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae400/0x85ace67feb16660b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a5a expref: 90 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.924497:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5ae400, rc 0) 00000100:00100000:1.0:1788572579.924565:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:37410:x1875453360698880:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 10722us (10923us total) trans 4294971111 rc 301/301 00000100:00100000:1.0:1788572579.924575:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5015 00000100:00100000:1.0:1788572579.929704:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.930234:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360699264 00000100:00100000:0.0:1788572579.930246:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360699264 00000100:00100000:0.0:1788572579.930330:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1093 00000100:00100000:0.0:1788572579.930342:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13fc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+89:37410:x1875453360699264:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572579.930743:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13fc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:37410:x1875453360699264:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 402us (1051us total) trans 4294971112 rc 0/0 00000100:00100000:0.0:1788572579.930751:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1093 00000100:00100000:0.0:1788572579.936506:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.936590:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360699392 00000100:00100000:2.0:1788572579.936602:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360699392 00000100:00100000:2.0:1788572579.936667:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5016 00000100:00100000:2.0:1788572579.936678:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+90:37410:x1875453360699392:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572579.936706:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572579.936731:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166620 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a61 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.936827:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166627 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.936844:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166627 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.936864:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166627 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572579.936906:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb5:0x0]: returning size 0 00000004:00200000:2.0:1788572579.936910:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572579.936931:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166627 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.936960:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af200/0x85ace67feb166620 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a61 expref: 92 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.936977:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166627 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a61 expref: 91 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.936987:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572579.936991:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572579.936996:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5afc00, rc 0) 00000100:00100000:2.0:1788572579.937105:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:37410:x1875453360699392:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 428us (610us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.937114:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5016 00000100:00100000:2.0:1788572579.941796:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572579.941937:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360699520 00000100:00100000:1.0:1788572579.941949:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360699520 00000100:00100000:1.0:1788572579.942074:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1465 00000100:00100000:1.0:1788572579.942086:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2348:x1875453360699520:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572579.942120:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572579.942148:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a68 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788572579.942182:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x61:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1788572579.942208:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=928s v=(0 0) 00010000:00010000:1.0:1788572579.942214:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572579.942225:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.942245:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 13 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572579.942272:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646a00, rc 0) 00000100:00100000:1.0:1788572579.942357:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c239aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2348:x1875453360699520:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 263us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572579.942364:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1465 00000100:00080000:0.0:1788572579.943735:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572579.947622:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572579.947965:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360699776 00000100:00100000:2.0:1788572579.947978:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360699776 00000100:00100000:2.0:1788572579.948086:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5017 00000100:00100000:2.0:1788572579.948096:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:37410:x1875453360699776:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572579.948239:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:37410:x1875453360699776:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 143us (628us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.948247:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5017 00000100:00100000:2.0:1788572579.956082:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572579.956386:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360699904 00000100:00100000:0.0:1788572579.956398:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360699904 00000100:00100000:0.0:1788572579.956460:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1466 00000100:00100000:0.0:1788572579.956470:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:37410:x1875453360699904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572579.956492:0:6681:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572579.956510:0:6681:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a6f expref: -99 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.956544:0:6681:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.956553:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572579.956557:0:6681:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572579.956561:0:6681:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.956596:0:6681:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 14 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572579.956610:0:6681:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 14 pid: 6682 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572579.956626:0:6681:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382412416:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572579.956703:0:6681:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572579.961988:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572579.962147:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700032 00000100:00100000:3.0:1788572579.962157:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700032 00010000:00010000:3.0:1788572579.962196:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 15 pid: 6682 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572579.962242:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1189 00000100:00100000:3.0:1788572579.962252:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360700032:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572579.962268:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572579.962295:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 15 pid: 6682 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.962308:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 15 pid: 6682 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.962332:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572579.962336:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572579.962355:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.962365:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572579.962409:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 15 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572579.962424:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382412544:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572579.962502:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734600 going to sleep for 11 seconds 00010000:00010000:2.0:1788572579.962597:0:6681:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb058e53/0x85ace67feb16662e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a68 expref: 15 pid: 6682 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572579.962612:0:6681:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6681:1875453382412416:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572579.962638:0:6681:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 14 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572579.962647:0:6681:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1a00, rc 0) 00000100:00100000:2.0:1788572579.962676:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2399500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:37410:x1875453360699904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6208us (6606us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572579.962684:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1466 00010000:00010000:1.0:1788572579.962861:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572579.976881:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5824:1875453382412544:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572579.976910:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572579.976986:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360700032:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 14735us (15005us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572579.976998:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1189 00000100:00100000:2.0:1788572580.000559:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572580.000664:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700160 00000100:00100000:0.0:1788572580.000673:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700160 00002000:00100000:0.0:1788572580.000728:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x61:0x0] (0->1048575) req@ffff9626c10a0e00 x1875453360700160/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:241/0 lens 488/0 e 0 to 0 dl 1788572591 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572580.000739:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a0e00 with x1875453360700160 ext(0->1048575) 00002000:00010000:0.0:1788572580.000746:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a0e00 00000100:00100000:0.0:1788572580.000782:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 382 00000100:00100000:0.0:1788572580.000791:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360700160:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572580.036610:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2349:x1875453360700160:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 35807us (36048us total) trans 4294967676 rc 0/0 00000100:00100000:3.0:1788572580.036624:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 382 00002000:00100000:3.0:1788572580.036636:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x61:0x0] (0->1048575) req@ffff9626c10a0e00 x1875453360700160/t4294967676(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:241/0 lens 488/448 e 0 to 0 dl 1788572591 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572580.036646:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c10a0e00 with x1875453360700160 ext(0->1048575) 00002000:00010000:3.0:1788572580.036653:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c10a0e00 00000100:00100000:1.0:1788572580.053550:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572580.053654:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700288 00000100:00100000:0.0:1788572580.053666:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700288 00000100:00100000:0.0:1788572580.053752:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5018 00000100:00100000:0.0:1788572580.053765:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+91:37410:x1875453360700288:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00080000:2.0:1788572580.053944:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1788572580.144909:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971112 is committed 00000001:00080000:3.0:1788572580.144927:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971100 is committed 00000004:00080000:3.0:1788572580.144946:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 44 committed 00000001:00080000:3.0:1788572580.144999:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971106 is committed 00000004:00080000:3.0:1788572580.145053:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 43 committed 00000004:00080000:3.0:1788572580.145076:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 48 committed 00000004:00080000:3.0:1788572580.145145:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 49 committed 00000100:00100000:3.0:1788572580.145440:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4921880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37410:x1875453360700288:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 91673us (91897us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.145453:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5018 00000100:00100000:3.0:1788572580.146049:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382413056:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1788572580.148479:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.148768:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572580.149529:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788572580.149545:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f596a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382412672:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1788572580.149602:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572580.149650:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700416 00000100:00100000:3.0:1788572580.149659:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700416 00000100:00100000:3.0:1788572580.149705:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382413056 00000100:00100000:3.0:1788572580.149707:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382413056 00000100:00100000:3.0:1788572580.149724:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382412672 00000100:00100000:3.0:1788572580.149727:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382412672 00000100:00100000:3.0:1788572580.149748:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1467 00000100:00100000:3.0:1788572580.149757:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2348:x1875453360700416:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1788572580.150393:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7875c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382412800:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1788572580.150435:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572580.150461:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1788572580.150467:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7877b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382412928:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1788572580.150489:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572580.150632:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4922680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382413184:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1788572580.150659:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572580.150739:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382412800 00000100:00100000:0.0:1788572580.150746:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382412800 00000100:00100000:0.0:1788572580.150788:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382412928 00000100:00100000:0.0:1788572580.150791:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382412928 00000100:00100000:0.0:1788572580.150802:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382413184 00000100:00100000:0.0:1788572580.150804:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382413184 00000100:00100000:0.0:1788572580.150847:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1468 00000100:00100000:0.0:1788572580.150858:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382413056:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1788572580.150878:0:6679:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:93 count 1 00010000:00010000:0.0:1788572580.150907:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.150930:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 84 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.150941:0:6679:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 10 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.150948:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=5s v=(1 0) 00010000:00010000:0.0:1788572580.150953:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572580.150959:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 5/0,1 mode: --/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.150989:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 10 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.151028:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 10 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.151040:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382413312:192.168.202.40@tcp:104: 00000100:00080000:2.0:1788572580.151246:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1788572580.151369:0:6679:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 84 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.151381:0:6679:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 84 pid: 10416 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.151392:0:6679:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382413440:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572580.151418:0:6679:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d13ebd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788572580.174311:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1469 00000100:00100000:3.0:1788572580.174333:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3302:x1875453382412672:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1788572580.174384:0:20911:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:94 count 1 00002000:00010000:3.0:1788572580.174426:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1788572580.174434:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.174452:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=928s v=(0 0) 00010000:00010000:3.0:1788572580.174456:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.174463:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.174478:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.174487:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.174500:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.174522:0:20911:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572580.175991:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382412672:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1658us (26379us total) trans 4294967676 rc 0/0 00000100:00100000:3.0:1788572580.176025:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1469 00000100:00100000:3.0:1788572580.176053:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1470 00000100:00100000:3.0:1788572580.176063:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3304:x1875453382412800:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1788572580.177269:0:20911:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:92 count 1 00010000:00010000:3.0:1788572580.177302:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177324:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177335:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 10 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177352:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=6s v=(1 0) 00010000:00010000:3.0:1788572580.177357:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.177362:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 5/0,1 mode: --/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177407:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 10 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177421:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 10 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.177437:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382413824:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.177538:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177548:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.177558:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ff0a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382413952:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572580.177581:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e77349c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572580.177615:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1471 00000100:00100000:3.0:1788572580.177629:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3304:x1875453382412928:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1788572580.177652:0:10416:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:95 count 1 00002000:00010000:3.0:1788572580.177680:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1788572580.177686:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177695:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=928s v=(0 0) 00010000:00010000:3.0:1788572580.177698:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.177704:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177725:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177735:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177744:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.177767:0:10416:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1788572580.178350:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382412928:12345-0@lo:6:osp-syn-1-0.0 Request processed in 714us (27847us total) trans 4294967677 rc 0/0 00000100:00100000:3.0:1788572580.178359:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1471 00000100:00100000:3.0:1788572580.178382:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1472 00000100:00100000:3.0:1788572580.178390:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1875453382413184:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1788572580.178406:0:10416:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:92 count 1 00010000:00010000:3.0:1788572580.178423:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178437:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 73 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178445:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 15 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178457:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=7s v=(1 0) 00010000:00010000:3.0:1788572580.178461:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.178465:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178491:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 15 pid: 6681 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178501:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 15 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.178513:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382414080:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.178562:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 73 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178572:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 73 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.178582:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382414208:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572580.178604:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734840 going to sleep for 11 seconds 00010000:00010000:3.0:1788572580.178629:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1788572580.178635:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178649:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572580.178654:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178662:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178679:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178687:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cff4d8e/0x85ace67feb166643 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178704:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572580.178707:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178713:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572580.178715:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178722:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178729:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178736:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000083ae6b3b/0x85ace67feb166651 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.178746:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.178748:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.178749:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.178756:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1788572580.178788:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f596a680/2, rc 0, transno 4294967676 00000100:00100000:3.0:1788572580.178803:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f596a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382412672:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1788572580.186597:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382413696:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1788572580.186691:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572580.186740:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788572580.186746:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c4920e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382413568:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1788572580.186766:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572580.186791:0:6681:0:(service.c:3462:ptlrpc_start_thread()) ost[0] started 6 min 3 max 116 00000100:00100000:2.0:1788572580.186800:0:6681:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost00_006' 00000004:00080000:2.0:1788572580.186947:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7877b80/2, rc 0, transno 4294967677 00000100:00100000:2.0:1788572580.186961:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7877b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382412928:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1788572580.186984:0:6682:0:(service.c:3462:ptlrpc_start_thread()) ost[0] started 6 min 3 max 116 00000100:00100000:2.0:1788572580.186992:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382413696 00000100:00100000:2.0:1788572580.186997:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382413696 00000100:00100000:2.0:1788572580.189498:0:6682:0:(service.c:3462:ptlrpc_start_thread()) ost[0] started 6 min 3 max 116 00000100:00100000:2.0:1788572580.189515:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382413568 00000100:00100000:2.0:1788572580.189523:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382413568 00000100:00100000:3.0:1788572580.190493:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.190519:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.190585:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.190606:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.190651:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.190676:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700672 00000100:00100000:3.0:1788572580.190682:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700672 00010000:00010000:3.0:1788572580.190726:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 85 pid: 10416 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.190750:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700544 00000100:00100000:3.0:1788572580.190753:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700544 00010000:00010000:3.0:1788572580.190769:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 11 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.190783:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700800 00000100:00100000:3.0:1788572580.190786:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700800 00010000:00010000:3.0:1788572580.190802:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 12 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.190822:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701056 00000100:00100000:3.0:1788572580.190825:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701056 00010000:00010000:3.0:1788572580.190839:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 16 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.190851:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360700928 00000100:00100000:3.0:1788572580.190853:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360700928 00010000:00010000:3.0:1788572580.190864:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 86 pid: 6679 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.190879:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1190 00000100:00100000:3.0:1788572580.190888:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2350:x1875453360700672:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.190904:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.190910:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 86 pid: 10416 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.190922:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 86 pid: 10416 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.190949:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.191001:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+86:2350:x1875453360700672:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 113us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.191028:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1190 00000100:00100000:3.0:1788572580.191047:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1191 00000100:00100000:3.0:1788572580.191054:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360700544:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.191062:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.191068:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 12 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191079:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 12 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191620:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.191626:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.191632:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191644:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191666:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.191692:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360700544:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 637us (1171us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.191700:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1191 00000100:00100000:3.0:1788572580.191723:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1192 00000100:00100000:3.0:1788572580.191730:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360700800:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.191741:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.191746:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 11 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191764:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 11 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191776:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.191794:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360700800:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 65us (1208us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.191801:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1192 00000100:00100000:3.0:1788572580.191827:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1193 00000100:00100000:3.0:1788572580.191835:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0faec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360701056:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.191846:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.191851:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 16 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191862:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 16 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191875:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.191894:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0faec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+16:2349:x1875453360701056:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 60us (1287us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.191900:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1193 00000100:00100000:3.0:1788572580.191942:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1194 00000100:00100000:3.0:1788572580.191949:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360700928:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.191957:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.191963:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 85 pid: 6679 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191973:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 85 pid: 6679 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.191987:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.191990:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.191995:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192030:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192045:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.192063:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360700928:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 114us (1412us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.192069:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1194 00010000:00010000:3.0:1788572580.192131:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000243e4f48/0x85ace67feb165e08 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1766 expref: 10 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.192147:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382413824:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.192159:0:20911:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008f131a0/0x85ace67feb165d60 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f16f6 expref: 84 pid: 6679 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.192169:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ff0a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20911:1875453382413952:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.192189:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192199:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192206:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192216:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192224:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192239:0:20911:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1788572580.192257:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572580.192264:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.192271:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572580.192275:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.192284:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.192300:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 1/0,0 mode: --/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.192306:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000064d42a5b/0x85ace67feb16664a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.192323:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572580.192325:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572580.192326:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572580.192788:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+9:3304:x1875453382412800:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16726us (42351us total) trans 4294967678 rc 0/0 00000100:00100000:3.0:1788572580.192796:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1470 00000100:00100000:3.0:1788572580.192813:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1473 00000100:00100000:3.0:1788572580.192819:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382413696:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1788572580.192841:0:20911:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:93 count 1 00010000:00010000:3.0:1788572580.192861:0:20911:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192874:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192883:0:20911:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192890:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=5s v=(1 0) 00010000:00010000:3.0:1788572580.192893:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.192898:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 5/0,1 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192924:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.192936:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.192949:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382414336:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.192998:0:20911:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.193037:0:20911:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.193047:0:20911:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382414464:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572580.193075:0:20911:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7734600 going to sleep for 11 seconds 00010000:00010000:3.0:1788572580.193101:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bac2935f/0x85ace67feb165d83 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1704 expref: 15 pid: 6681 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.193112:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382414080:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572580.193123:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627d131f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382414208:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.193140:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1788572580.193165:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:2.0:1788572580.193198:0:3304:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627e7875c00/2, rc 0, transno 4294967678 00000100:00100000:2.0:1788572580.193209:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7875c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382412800:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1788572580.193263:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1474 00000100:00100000:3.0:1788572580.193271:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+7:3305:x1875453382413568:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1788572580.193286:0:23814:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:95 count 1 00002000:00010000:3.0:1788572580.193309:0:23814:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1788572580.193314:0:23814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.193323:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=928s v=(0 0) 00010000:00010000:3.0:1788572580.193326:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.193330:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.193354:0:23814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.193362:0:23814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.193373:0:23814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.193392:0:23814:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572580.193802:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+8:3305:x1875453382413568:12345-0@lo:6:osp-syn-0-0.0 Request processed in 530us (7034us total) trans 4294967677 rc 0/0 00000100:00100000:3.0:1788572580.193809:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1474 00000004:00080000:3.0:1788572580.193841:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9626c4920e00/2, rc 0, transno 4294967677 00000100:00100000:3.0:1788572580.193852:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4920e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382413568:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1788572580.194187:0:5825:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572580.194194:0:5825:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.194202:0:5825:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572580.194206:0:5825:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.194215:0:5825:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.194230:0:5825:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.194241:0:5825:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb711951/0x85ace67feb166666 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.194258:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572580.194260:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572580.194262:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572580.195781:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007278c512/0x85ace67feb165fd6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1815 expref: 9 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.195799:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382413312:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572580.195827:0:6679:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000309a3bed/0x85ace67feb165f20 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f17cf expref: 83 pid: 10416 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.195843:0:6679:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6679:1875453382413440:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572580.195901:0:6679:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.195914:0:6679:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.195922:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.195933:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.195942:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.195962:0:6679:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572580.196556:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3302:x1875453382413056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 45698us (47786us total) trans 4294967679 rc 0/0 00000100:00100000:0.0:1788572580.196565:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1468 00010000:00010000:3.0:1788572580.197543:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572580.197553:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.197564:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572580.197568:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.197578:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.197597:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 1/0,0 mode: --/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.197603:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024203c19/0x85ace67feb16663c lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.197630:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.197632:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.197634:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1788572580.197674:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627fb1e3800/2, rc 0, transno 4294967679 00000100:00100000:3.0:1788572580.197690:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb1e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382413056:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1788572580.199722:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.206361:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701184 00000100:00100000:3.0:1788572580.206379:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701184 00010000:00010000:3.0:1788572580.206453:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 74 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.206498:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1195 00000100:00100000:3.0:1788572580.206507:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2349:x1875453360701184:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.206523:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.206528:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 74 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206541:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 74 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206564:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba018e50/0x85ace67feb165cbf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1686 expref: 74 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206582:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.206586:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.206593:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206602:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206630:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.206684:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360701184:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 178us (6971us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.206692:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1195 00000100:00080000:3.0:1788572580.206716:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1788572580.206729:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206738:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206748:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206755:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.206773:0:10416:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572580.208166:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.209467:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+8:3303:x1875453382413184:12345-0@lo:6:osp-syn-0-0.0 Request processed in 31069us (58798us total) trans 4294967678 rc 0/0 00000100:00100000:3.0:1788572580.209481:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1472 00010000:00010000:3.0:1788572580.209525:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572580.209533:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.209545:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572580.209551:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.209562:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.209583:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.209592:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166658 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.209615:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.209617:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.209618:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572580.209660:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382414336:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572580.209674:0:20911:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1b59c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20911:1875453382414464:192.168.202.40@tcp:104: 00000004:00080000:1.0:1788572580.211464:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9626c4922680/2, rc 0, transno 4294967678 00000100:00100000:1.0:1788572580.211487:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c4922680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382413184:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1788572580.211572:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00010000:00010000:3.0:1788572580.212228:0:20911:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.212655:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701312 00000100:00100000:3.0:1788572580.212664:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701312 00010000:00010000:3.0:1788572580.212708:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.212741:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701440 00000100:00100000:3.0:1788572580.212744:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701440 00010000:00010000:3.0:1788572580.212764:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.212789:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1196 00000100:00100000:3.0:1788572580.212798:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360701312:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.213545:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.213556:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.213571:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.213598:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb4113a9/0x85ace67feb165fcf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f180e expref: 15 pid: 20911 timeout: 1028 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.213628:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.213707:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360701312:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 909us (5547us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.213796:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1196 00000100:00100000:1.0:1788572580.215200:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1197 00000100:00100000:1.0:1788572580.215225:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360701440:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572580.215245:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572580.215255:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.215274:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.215295:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000888c8b98/0x85ace67feb165e7f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1790 expref: 73 pid: 6682 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.215312:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572580.215317:0:11655:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572580.215322:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.215351:0:11655:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.215394:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572580.215518:0:20911:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.215529:0:20911:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.215542:0:20911:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.215550:0:20911:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.215571:0:20911:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572580.216377:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+8:3305:x1875453382413696:12345-0@lo:6:osp-syn-0-0.0 Request processed in 23552us (29681us total) trans 4294967679 rc 0/0 00000100:00100000:0.0:1788572580.216387:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1473 00000100:00100000:1.0:1788572580.216472:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2349:x1875453360701440:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 226us (3871us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572580.216483:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1197 00010000:00010000:3.0:1788572580.218485:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572580.218499:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.218514:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572580.218520:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.218531:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.218551:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 1/0,0 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.218559:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004f071138/0x85ace67feb16665f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.218598:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.218600:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572580.218601:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1788572580.218646:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff962801f91880/2, rc 0, transno 4294967679 00000100:00100000:3.0:1788572580.218655:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382413696:0@lo:6:osp-syn-0-0.0 00000001:00080000:2.0:1788572580.259153:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967676 is committed 00000100:00100000:1.0:1788572580.259346:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360700416:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 109573us (110860us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572580.259358:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1467 00000100:00100000:3.0:1788572580.264395:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572580.265162:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701696 00000100:00100000:2.0:1788572580.265174:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701696 00000100:00100000:2.0:1788572580.265249:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5019 00000100:00100000:2.0:1788572580.265260:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37410:x1875453360701696:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572580.265323:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572580.265366:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166627 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a61 expref: 60 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.265386:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572580.265393:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572580.265429:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.265445:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.265455:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572580.266271:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005a66a102 reply state 00000000230fa4c6 transno 4294971113 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572580.266283:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572580.266445:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37410:x1875453360701696:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1185us (2058us total) trans 4294971113 rc 0/0 00000100:00100000:2.0:1788572580.266452:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5019 00010000:00010000:1.0:1788572580.281529:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.281552:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.281561:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.281567:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff5aea00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.281571:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.281580:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.281598:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:1.0:1788572580.281601:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572580.281606:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.281613:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5aea00/0x85ace67feb16666d lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.285357:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572580.285800:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701824 00000100:00100000:2.0:1788572580.285811:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701824 00000100:00100000:2.0:1788572580.285872:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5020 00000100:00100000:2.0:1788572580.285883:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37410:x1875453360701824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572580.285909:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572580.285934:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a76 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.286028:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.286044:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.286066:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572580.286070:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572580.286073:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff5afc00, rc 0) 00000100:00100000:2.0:1788572580.286145:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37410:x1875453360701824:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 263us (798us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572580.286152:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5020 00000100:00100000:0.0:1788572580.293354:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572580.295231:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360701952 00000100:00100000:2.0:1788572580.295248:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360701952 00000100:00100000:2.0:1788572580.295327:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1475 00000100:00100000:2.0:1788572580.295348:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360701952:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1788572580.295576:0:6682:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x61:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:2.0:1788572580.295580:0:6682:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x61:0x0] updating lvb mtime from disk: 0 -> 1788572576 00002000:00010000:2.0:1788572580.295582:0:6682:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x61:0x0] updating lvb atime from disk: 0 -> 1788572576 00002000:00010000:2.0:1788572580.295585:0:6682:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x61:0x0] updating lvb ctime from disk: 0 -> 1788572576 00002000:00010000:2.0:1788572580.295587:0:6682:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x61:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:2.0:1788572580.295663:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360701952:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 317us (2332us total) trans 4294967680 rc 0/0 00000100:00100000:2.0:1788572580.295671:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1475 00000100:00100000:0.0:1788572580.302473:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572580.302641:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360702208 00000100:00100000:1.0:1788572580.302653:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360702208 00000100:00100000:1.0:1788572580.302718:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5021 00000100:00100000:1.0:1788572580.302729:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37410:x1875453360702208:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572580.302824:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.302843:0:23058:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.302854:0:23058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 4/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.302894:0:23058:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.302908:0:23058:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 61 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572580.302922:0:23058:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382414592:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572580.303105:0:23058:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572580.307256:0:23058:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:23058:1875453382414592:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572580.307289:0:23058:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.308736:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.309350:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360702336 00000100:00100000:3.0:1788572580.309362:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360702336 00010000:00010000:3.0:1788572580.309406:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.309450:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1198 00000100:00100000:3.0:1788572580.309461:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2349:x1875453360702336:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.309479:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.309486:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.309500:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.309523:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5ae800/0x85ace67feb1665f6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb5:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a4c expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.309539:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:3.0:1788572580.309545:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.309556:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.309564:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.309602:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.309660:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2349:x1875453360702336:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 201us (932us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.309669:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1198 00010000:00010000:2.0:1788572580.310382:0:23058:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310393:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310423:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310431:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310438:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310444:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572580.310449:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310456:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310469:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:2.0:1788572580.310472:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572580.310476:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.310482:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647e00/0x85ace67feb16667b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572580.310552:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37410:x1875453360702208:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 7823us (8088us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572580.310561:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5021 00000100:00100000:0.0:1788572580.323763:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.323867:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360702464 00000100:00100000:3.0:1788572580.323877:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360702464 00000100:00100000:3.0:1788572580.323943:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1476 00000100:00100000:3.0:1788572580.323963:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2347:x1875453360702464:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572580.324056:0:23814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.324782:0:23814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 12 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.324795:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572580.324800:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572580.324805:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.324912:0:23814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 12 pid: 6681 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.324927:0:23814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 12 pid: 6681 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.324942:0:23814:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:23814:1875453382414720:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572580.325075:0:23814:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627e7495000 going to sleep for 11 seconds 00000100:00100000:3.0:1788572580.336866:0:23814:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c121f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:23814:1875453382414720:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572580.336905:0:23814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.339667:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.339859:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360702592 00000100:00100000:3.0:1788572580.339870:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360702592 00010000:00010000:3.0:1788572580.339916:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 13 pid: 6681 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572580.339976:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1199 00000100:00100000:3.0:1788572580.339987:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2350:x1875453360702592:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572580.340022:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572580.340060:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 13 pid: 6681 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.340088:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 13 pid: 6681 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.340114:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b75b8bf/0x85ace67feb166635 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a6f expref: 13 pid: 6681 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.340132:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572580.340136:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572580.340142:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.340153:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.340188:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572580.340248:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360702592:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 263us (594us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.340256:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1199 00010000:00010000:2.0:1788572580.340376:0:23814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340386:0:23814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340409:0:23814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340416:0:23814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340424:0:23814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340431:0:23814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572580.340436:0:23814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340445:0:23814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340457:0:23814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.340464:0:23814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab8b6c39/0x85ace67feb166682 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572580.340516:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360702464:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 16554us (16764us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572580.340524:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1476 00000100:00100000:0.0:1788572580.344967:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572580.345459:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360702848 00000100:00100000:1.0:1788572580.345470:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360702848 00000100:00100000:1.0:1788572580.345534:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5022 00000100:00100000:1.0:1788572580.345544:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37410:x1875453360702848:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1788572580.345638:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.345658:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.345671:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.345898:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.345910:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.345918:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.346223:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.346240:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 61 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572580.346256:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382414848:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572580.346351:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572580.350647:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572580.350999:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360702976 00000100:00100000:0.0:1788572580.351051:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360702976 00010000:00010000:0.0:1788572580.351102:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572580.351141:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1200 00000100:00100000:0.0:1788572580.351151:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2349:x1875453360702976:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572580.351167:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572580.351173:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.351186:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 62 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.351208:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:0.0:1788572580.351213:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.351223:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.351232:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.351252:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572580.351311:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2349:x1875453360702976:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 162us (674us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572580.351318:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1200 00010000:00010000:1.0:1788572580.352792:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5afc00/0x85ace67feb166674 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a76 expref: 61 pid: 5838 timeout: 1028 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572580.352827:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382414848:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572580.352861:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.352872:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.352878:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572580.353341:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971114 00000001:00080000:1.0:1788572580.353511:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971115 00000001:00080000:1.0:1788572580.353562:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971116 00000100:00100000:1.0:1788572580.353651:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382414976:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572580.353719:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572580.353805:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572580.359929:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382414976 00000100:00100000:3.0:1788572580.359945:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382414976 00000100:00100000:3.0:1788572580.360056:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1788572580.360068:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:5837:x1875453382414976:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572580.360491:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:5837:x1875453382414976:12345-0@lo:1000:mdt00_000.0 Request processed in 425us (6770us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.360502:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1788572580.360588:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382414976:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1788572580.361417:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971117 00000100:00100000:1.0:1788572580.361504:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382415104:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572580.361549:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572580.361574:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572580.361654:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382415104 00000100:00100000:0.0:1788572580.361660:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382415104 00000100:00100000:0.0:1788572580.361713:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:0.0:1788572580.361721:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:5837:x1875453382415104:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572580.361905:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:5837:x1875453382415104:12345-0@lo:1000:mdt00_000.0 Request processed in 184us (355us total) trans 4294967681 rc 0/0 00000100:00100000:0.0:1788572580.361913:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:1.0:1788572580.362612:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382415104:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1788572580.362715:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971118 00010000:00010000:1.0:1788572580.362750:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362761:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362769:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362775:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.362780:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362789:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362816:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627e3306400) 00010000:00010000:1.0:1788572580.362819:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572580.362824:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362829:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166690 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb5:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362861:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362867:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362872:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362876:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.362879:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362885:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362892:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572580.362895:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572580.362901:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 241 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362912:0:5837:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 241 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362920:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 241 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.362961:0:5837:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff5af400/0x85ace67feb16659b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a30 expref: 241 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572580.362975:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382415232:192.168.202.40@tcp:105: 00000100:00100000:1.0:1788572580.363385:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572580.371375:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572580.371729:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382415232:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572580.371759:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371770:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16657f lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371819:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371833:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371845:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371856:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371862:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371868:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371874:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.371878:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371884:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371895:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572580.371897:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572580.371901:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371907:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb166697 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371916:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371921:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371927:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371931:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.371934:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371939:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371945:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb0:0x0].0x0 (ffff9627ff5abc00) 00010000:00010000:1.0:1788572580.371958:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572580.371961:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.371966:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb166689 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb0:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572580.372447:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37410:x1875453360702848:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 26901us (27499us total) trans 4294971119 rc 0/0 00000100:00100000:1.0:1788572580.372458:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5022 00000020:00080000:1.0:1788572580.372463:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:2.0:1788572580.372560:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360703104 00000100:00100000:2.0:1788572580.372568:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360703104 00000100:00100000:2.0:1788572580.372623:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1094 00000100:00100000:2.0:1788572580.372633:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:37407:x1875453360703104:12345-192.168.202.40@tcp:49:md5sum.0 00000100:00100000:2.0:1788572580.372730:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:37407:x1875453360703104:12345-192.168.202.40@tcp:49:md5sum.0 Request processed in 98us (1364us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572580.372736:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1094 00000040:00080000:0.0:1788572580.372959:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572580.372965:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572580.373736:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 7 00000040:00080000:0.0:1788572580.373746:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572580.378326:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572580.378433:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360703232 00000100:00100000:0.0:1788572580.378443:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360703232 00000100:00100000:0.0:1788572580.378505:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1477 00000100:00100000:0.0:1788572580.378515:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:37407:x1875453360703232:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572580.378535:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572580.378557:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000532f215a/0x85ace67feb16669e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a7d expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572580.378583:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x61:0x0] initial LVB size: 1048576, mtime: 0x6a9b73a0, atime: 0x6a9b73a0, ctime: 0x6a9b73a0, blocks: 0x803 00010000:00010000:0.0:1788572580.378609:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=928s v=(0 0) 00010000:00010000:0.0:1788572580.378614:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572580.378623:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000532f215a/0x85ace67feb16669e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a7d expref: 73 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.378640:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000532f215a/0x85ace67feb16669e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a7d expref: 73 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.378652:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1000, rc 0) 00000100:00100000:0.0:1788572580.378716:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:37407:x1875453360703232:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 202us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572580.378724:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1477 00000100:00100000:3.0:1788572580.381586:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.381681:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572580.382190:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360703616 00000100:00100000:2.0:1788572580.382200:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360703616 00002000:00100000:2.0:1788572580.382253:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x61:0x0] (0->32767) req@ffff9626c1b5a680 x1875453360703616/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:241/0 lens 488/0 e 0 to 0 dl 1788572591 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572580.382272:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b5a680 with x1875453360703616 ext(0->32767) 00002000:00010000:2.0:1788572580.382278:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b5a680 00000100:00100000:2.0:1788572580.382309:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 383 00000100:00100000:2.0:1788572580.382319:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360703616:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:1.0:1788572580.383741:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360703488 00000100:00100000:1.0:1788572580.383751:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360703488 00000100:00100000:1.0:1788572580.383835:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1095 00000100:00100000:1.0:1788572580.383850:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37410:x1875453360703488:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1788572580.388825:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xb5:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1788572580.389075:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971120 00010000:00010000:1.0:1788572580.389299:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.389317:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.389338:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.389348:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.389354:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.389361:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.389389:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788572580.389597:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572580.389605:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.389612:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572580.389616:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.389625:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.389638:0:7616:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb5:0x0].0x0 (ffff9627ff5ab800) 00010000:00010000:0.0:1788572580.389641:0:7616:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572580.389646:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 1/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.389652:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646c00/0x85ace67feb1666a5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572580.389669:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572580.389671:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572580.389673:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572580.390389:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37410:x1875453360703488:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 6538us (8805us total) trans 4294971121 rc 0/0 00000100:00100000:1.0:1788572580.390402:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1095 00000020:00080000:1.0:1788572580.390408:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788572580.396844:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360703616:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 14515us (15151us total) trans 0 rc 32768/32768 00000100:00100000:1.0:1788572580.396866:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 383 00002000:00100000:1.0:1788572580.396881:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x61:0x0] (0->32767) req@ffff9626c1b5a680 x1875453360703616/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:241/0 lens 488/440 e 0 to 0 dl 1788572591 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572580.396890:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c1b5a680 with x1875453360703616 ext(0->32767) 00002000:00010000:1.0:1788572580.396897:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c1b5a680 00000100:00100000:3.0:1788572580.409711:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572580.409797:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360703744 00000100:00100000:1.0:1788572580.409817:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360703744 00002000:00100000:1.0:1788572580.409867:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x61:0x0] (32768->1048575) req@ffff9627d131f100 x1875453360703744/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:241/0 lens 488/0 e 0 to 0 dl 1788572591 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572580.409879:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131f100 with x1875453360703744 ext(32768->1048575) 00002000:00010000:1.0:1788572580.409885:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131f100 00000100:00100000:1.0:1788572580.409916:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 384 00000100:00100000:1.0:1788572580.409925:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2347:x1875453360703744:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:1.0:1788572580.451813:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2347:x1875453360703744:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 41870us (42093us total) trans 0 rc 1015808/1015808 00000100:00100000:1.0:1788572580.451827:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 384 00002000:00100000:1.0:1788572580.451841:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x61:0x0] (32768->1048575) req@ffff9627d131f100 x1875453360703744/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:241/0 lens 488/440 e 0 to 0 dl 1788572591 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572580.451851:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627d131f100 with x1875453360703744 ext(32768->1048575) 00002000:00010000:1.0:1788572580.451859:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627d131f100 00000100:00100000:0.0:1788572580.466200:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.466444:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360703872 00000100:00100000:3.0:1788572580.466455:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360703872 00000100:00100000:3.0:1788572580.466521:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 430 00000100:00100000:3.0:1788572580.466531:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:2350:x1875453360703872:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572580.466574:0:5842:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572580.466596:0:5842:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a84 expref: -99 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.466657:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.466671:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.466683:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.466700:0:5842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.466708:0:5842:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.466714:0:5842:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467357:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467368:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467374:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467380:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572580.467384:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467391:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467404:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572580.467407:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572580.467412:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467418:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb1666b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467430:0:5842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467436:0:5842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467441:0:5842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467448:0:5842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572580.467451:0:5842:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467456:0:5842:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467463:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x8799b2a3 (ffff9626c4b10600) 00010000:00010000:3.0:1788572580.467466:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572580.467468:0:5842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467474:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0x81:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467511:0:5842:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666ac lrc: 1/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a84 expref: 243 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467521:0:5842:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x81:0x0].0x0 (ffff9627c7d65400) 00010000:00010000:3.0:1788572580.467524:0:5842:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572580.467528:0:5842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x81:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a84 expref: 243 pid: 5842 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.467537:0:5842:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4a00, rc -2) 00000100:00100000:2.0:1788572580.467714:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:2350:x1875453360703872:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1182us (1522us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572580.467723:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 430 00000100:00100000:2.0:1788572580.471655:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.472186:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360704000 00000100:00100000:3.0:1788572580.472198:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360704000 00000100:00100000:3.0:1788572580.472258:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 431 00000100:00100000:3.0:1788572580.472269:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:2348:x1875453360704000:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:3.0:1788572580.472320:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572580.472355:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1666c1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a8b expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472429:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472446:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472459:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472473:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472479:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472486:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472805:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472815:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472822:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472827:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572580.472831:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472838:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472850:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572580.472853:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572580.472858:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472865:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666c8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472876:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472882:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472887:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472891:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572580.472894:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472899:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472905:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x8799b2a3 (ffff9626c4b10d00) 00010000:00010000:3.0:1788572580.472907:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572580.472910:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472916:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472949:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1666c1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a8b expref: 243 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472959:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffff9627c8ed8900) 00010000:00010000:3.0:1788572580.472961:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788572580.472965:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb1666c1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a8b expref: 243 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.472974:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5000, rc -2) 00000100:00100000:3.0:1788572580.473070:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:2348:x1875453360704000:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 802us (1427us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572580.473080:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 431 00000100:00100000:1.0:1788572580.493703:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572580.493815:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360704256 00000100:00100000:0.0:1788572580.493825:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360704256 00000100:00100000:0.0:1788572580.493894:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1096 00000100:00100000:0.0:1788572580.493904:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+242:37407:x1875453360704256:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:0.0:1788572580.494484:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37407:x1875453360704256:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 579us (789us total) trans 4294971122 rc 0/0 00000100:00100000:0.0:1788572580.494492:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1096 00000100:00100000:0.0:1788572580.557439:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.557558:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360704512 00000100:00100000:3.0:1788572580.557568:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360704512 00000100:00100000:3.0:1788572580.557636:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5023 00000100:00100000:3.0:1788572580.557647:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37412:x1875453360704512:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572580.557674:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572580.557698:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a92 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.557784:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.557817:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.557829:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.557845:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.558210:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.558218:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561186:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1666eb lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561259:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1666eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561272:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1666eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572580.561289:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb1:0x0]: returning size 0 00010000:00010000:2.0:1788572580.561422:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561430:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561437:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561442:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572580.561447:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561455:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561467:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572580.561471:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572580.561475:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561481:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb1666dd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561495:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561501:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561506:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561511:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572580.561572:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561580:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561587:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x4009f06f (ffff9626c4b10d00) 00010000:00010000:2.0:1788572580.561589:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572580.561592:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561598:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1666e4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x4009f06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561928:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb1666d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a92 expref: 246 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561946:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1666eb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a92 expref: 245 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.561954:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306600) 00010000:00010000:2.0:1788572580.561956:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572580.561959:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9000, rc 0) 00000100:00100000:2.0:1788572580.562210:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c121f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37412:x1875453360704512:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 4562us (4779us total) trans 4294971123 rc 0/0 00000100:00100000:2.0:1788572580.562219:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5023 00000100:00100000:3.0:1788572580.573743:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572580.574055:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360704768 00000100:00100000:1.0:1788572580.574065:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360704768 00000100:00100000:1.0:1788572580.574144:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 381 00000100:00100000:1.0:1788572580.574154:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37412:x1875453360704768:12345-192.168.202.40@tcp:103:md5sum.0 00010000:00010000:1.0:1788572580.574174:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572580.574209:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb1666eb lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a92 expref: 245 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.574230:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9627e3306600) 00010000:00010000:1.0:1788572580.574241:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572580.574287:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d131ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37412:x1875453360704768:12345-192.168.202.40@tcp:103:md5sum.0 Request processed in 134us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572580.574294:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 381 00000100:00100000:1.0:1788572580.576244:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572580.576388:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360704896 00000100:00100000:3.0:1788572580.576397:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360704896 00000100:00100000:3.0:1788572580.576464:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5024 00000100:00100000:3.0:1788572580.576474:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37412:x1875453360704896:12345-192.168.202.40@tcp:58:md5sum.0 00010000:00010000:3.0:1788572580.576601:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.576621:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572580.576635:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572580.576968:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].175, 136 off31784 00000100:00100000:3.0:1788572580.577097:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37412:x1875453360704896:12345-192.168.202.40@tcp:58:md5sum.0 Request processed in 623us (859us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572580.577105:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5024 00000100:00100000:0.0:1788572580.579822:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572580.579901:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705024 00000100:00100000:2.0:1788572580.579911:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705024 00000100:00100000:2.0:1788572580.579967:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5025 00000100:00100000:2.0:1788572580.579977:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37412:x1875453360705024:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572580.579995:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572580.580046:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1a99 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.580112:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.580126:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 4/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 245 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.580144:0:5839:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.580156:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 245 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.580169:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10600) 00010000:00010000:2.0:1788572580.580172:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572580.580176:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 245 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572580.580184:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9600, rc 0) 00000100:00100000:2.0:1788572580.580730:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:37412:x1875453360705024:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 753us (915us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572580.580739:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5025 00000100:00100000:0.0:1788572580.977497:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572580.978063:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705152 00000100:00100000:1.0:1788572580.978074:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705152 00000100:00100000:1.0:1788572580.978138:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 432 00000100:00100000:1.0:1788572580.978929:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:2350:x1875453360705152:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:1.0:1788572580.978969:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572580.978990:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166707 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1aa0 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979091:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979108:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979123:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979139:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979473:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979481:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979929:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979938:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979945:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979950:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.979955:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979962:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979975:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572580.979978:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572580.979983:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.979989:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb16670e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980001:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980071:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980077:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980081:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572580.980083:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980089:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980095:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x8799b2a3 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572580.980097:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572580.980100:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.980105:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166715 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6b:0x0].0x8799b2a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.982123:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166707 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aa0 expref: 246 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.982812:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6b:0x0].0x0 (ffff9627d1866300) 00010000:00010000:1.0:1788572580.982816:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788572580.982827:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166707 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aa0 expref: 246 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572580.982852:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646600, rc -2) 00000100:00100000:1.0:1788572580.983351:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+245:2350:x1875453360705152:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 5202us (5861us total) trans 0 rc -2/-2 00000100:00100000:1.0:1788572580.983363:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 432 00010000:00010000:2.0:1788572580.989627:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1788572581.434422:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572581.434432:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572581.434477:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 176 count 8 00000040:00080000:0.0:1788572581.434484:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1788572581.434615:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbc380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382415360:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572581.434726:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c4c47cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572581.452837:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.452946:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbc380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382415360:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1788572581.452998:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705280 00000100:00100000:1.0:1788572581.453056:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705280 00000100:00100000:1.0:1788572581.453131:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5026 00000100:00100000:1.0:1788572581.453144:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37414:x1875453360705280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572581.453188:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb1:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572581.453310:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37414:x1875453360705280:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 169us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.453318:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5026 00000100:00100000:1.0:1788572581.465096:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.465276:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705408 00000100:00100000:3.0:1788572581.465512:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705408 00000100:00100000:3.0:1788572581.465617:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5027 00000100:00100000:3.0:1788572581.465631:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37414:x1875453360705408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572581.465751:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465776:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465792:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465810:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465819:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465825:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465856:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465866:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465872:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572581.465923:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb1:0x0]: returning size 0 00000004:00200000:3.0:1788572581.465929:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572581.465947:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465961:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465968:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465975:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465981:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572581.465986:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.465995:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466046:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572581.466050:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572581.466057:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466064:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16671c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466083:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466094:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466101:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466106:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572581.466110:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466116:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466123:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x401fd760 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572581.466126:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572581.466129:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466135:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166723 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x401fd760 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466151:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466156:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466162:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466167:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572581.466170:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466176:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466183:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10600) 00010000:00010000:3.0:1788572581.466185:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572581.466188:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.466194:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb16672a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572581.466312:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37414:x1875453360705408:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 682us (1226us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.466324:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5027 00000100:00100000:0.0:1788572581.472938:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.473117:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705664 00000100:00100000:1.0:1788572581.473127:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705664 00000100:00100000:1.0:1788572581.473187:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5028 00000100:00100000:1.0:1788572581.473198:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37414:x1875453360705664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572581.473227:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572581.473250:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166731 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1aa7 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473370:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473386:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473398:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572581.473410:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb1:0x0]: returning size 0 00010000:00010000:1.0:1788572581.473520:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473528:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473535:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473541:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572581.473546:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473553:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473565:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10600) 00010000:00010000:1.0:1788572581.473568:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788572581.473573:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473580:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb166738 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473624:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166731 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aa7 expref: 69 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473640:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:1.0:1788572581.473642:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572581.473646:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166731 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aa7 expref: 69 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.473655:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646600, rc 0) 00000100:00100000:1.0:1788572581.473723:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37414:x1875453360705664:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 527us (799us total) trans 4294971124 rc 301/301 00000100:00100000:1.0:1788572581.473731:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5028 00000100:00100000:0.0:1788572581.480913:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.481049:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705792 00000100:00100000:3.0:1788572581.481064:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705792 00000100:00100000:3.0:1788572581.481145:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5029 00000100:00100000:3.0:1788572581.481158:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37414:x1875453360705792:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1788572581.481213:0:9040:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb1:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1788572581.481393:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37414:x1875453360705792:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 237us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.481402:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5029 00000100:00100000:2.0:1788572581.491261:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.491313:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.491340:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.491363:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.491381:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.491402:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.491437:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360706176 00000100:00100000:2.0:1788572581.491447:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360706176 00000100:00100000:2.0:1788572581.491514:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360706432 00000100:00100000:2.0:1788572581.491518:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360706432 00000100:00100000:2.0:1788572581.491549:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360706304 00000100:00100000:1.0:1788572581.491550:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360706048 00000100:00100000:2.0:1788572581.491554:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360706304 00000100:00100000:1.0:1788572581.491566:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360706048 00000100:00100000:2.0:1788572581.491600:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 631 00000100:00100000:2.0:1788572581.491617:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2348:x1875453360706176:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572581.491629:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 632 00000100:00100000:1.0:1788572581.491644:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360706432:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572581.491699:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:2348:x1875453360706176:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 88us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572581.491708:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 631 00000100:00100000:1.0:1788572581.491738:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360706432:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 93us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572581.491741:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 633 00000100:00100000:1.0:1788572581.491759:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 632 00000100:00100000:2.0:1788572581.491767:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360706304:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572581.491801:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 634 00000100:00100000:2.0:1788572581.491809:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2348:x1875453360706304:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 52us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.491818:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360706048:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572581.491828:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 633 00000100:00080000:2.0:1788572581.491860:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572581.491867:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7b9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360706048:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 52us (497us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.491878:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 634 00000100:00100000:2.0:1788572581.491880:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360706688 00000100:00100000:2.0:1788572581.491887:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360706688 00000100:00100000:2.0:1788572581.491935:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5030 00000100:00100000:2.0:1788572581.491945:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37414:x1875453360706688:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572581.491961:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572581.491986:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166746 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1aae expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.492599:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb16674d lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.492620:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb16674d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.492633:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb16674d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572581.492646:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb6:0x0]: returning size 0 00010000:00010000:2.0:1788572581.492696:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166746 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aae expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.492714:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb16674d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aae expref: 70 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.492724:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9627e3306200) 00010000:00010000:2.0:1788572581.492727:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572581.492731:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9000, rc 0) 00000100:00100000:2.0:1788572581.492787:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37414:x1875453360706688:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 844us (1382us total) trans 4294971125 rc 0/0 00000100:00100000:2.0:1788572581.492795:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5030 00000100:00100000:3.0:1788572581.493087:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360705920 00000100:00100000:3.0:1788572581.493097:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360705920 00000100:00100000:3.0:1788572581.493149:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 693 00000100:00100000:3.0:1788572581.493158:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0f1d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360705920:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572581.493203:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0f1d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360705920:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 47us (1953us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.493211:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 693 00000100:00100000:2.0:1788572581.505906:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.506390:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360706944 00000100:00100000:0.0:1788572581.506402:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360706944 00000100:00100000:0.0:1788572581.506457:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5031 00000100:00100000:0.0:1788572581.506468:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37414:x1875453360706944:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572581.506529:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572581.506557:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb16674d lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aae expref: 70 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.506578:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9627e3306200) 00010000:00010000:0.0:1788572581.506591:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572581.506615:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.506630:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.506641:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.506656:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.506664:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.506671:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507171:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507180:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507188:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507194:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572581.507401:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507411:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507422:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572581.507425:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572581.507429:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507436:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb16675b lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507448:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507455:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507461:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507467:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572581.507470:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507477:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507484:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xebd7e624 (ffff9627c5160300) 00010000:00010000:0.0:1788572581.507487:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572581.507490:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.507496:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166762 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xebd7e624 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.507608:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37414:x1875453360706944:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1138us (1710us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572581.507617:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5031 00000100:00100000:1.0:1788572581.511664:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.511814:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360707072 00000100:00100000:3.0:1788572581.511831:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360707072 00000100:00100000:3.0:1788572581.511924:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5032 00000100:00100000:3.0:1788572581.511937:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37414:x1875453360707072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572581.511967:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572581.512001:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb166769 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1ab5 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.512099:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.512117:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.512132:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.512270:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4600/0x85ace67feb166769 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.512305:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.512315:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572581.512318:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572581.512390:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e5800, rc 0) 00000100:00100000:3.0:1788572581.512618:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37414:x1875453360707072:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 682us (960us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.512627:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5032 00000100:00100000:2.0:1788572581.520119:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.520707:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360707328 00000100:00100000:1.0:1788572581.520722:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360707328 00000100:00100000:1.0:1788572581.520814:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5033 00000100:00100000:1.0:1788572581.520826:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37414:x1875453360707328:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1788572581.520944:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.520969:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.520985:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788572581.521301:0:23058:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009bad4a01 reply state 00000000ff3bf339 transno 4294971126 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572581.521312:0:23058:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788572581.523173:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37414:x1875453360707328:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2331us (3051us total) trans 4294971126 rc 0/0 00000100:00100000:1.0:1788572581.523191:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5033 00010000:00010000:1.0:1788572581.527497:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.527521:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.527545:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.527552:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572581.527559:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.527572:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.527596:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:1.0:1788572581.527599:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572581.527606:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.527612:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb166777 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572581.545085:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.545195:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360707456 00000100:00100000:1.0:1788572581.545208:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360707456 00000100:00100000:1.0:1788572581.545276:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5034 00000100:00100000:1.0:1788572581.545301:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:37414:x1875453360707456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572581.545342:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572581.545371:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1abc expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.545459:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.545478:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.545489:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:1.0:1788572581.545493:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572581.545498:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646600, rc 0) 00000100:00100000:1.0:1788572581.545588:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37414:x1875453360707456:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 302us (518us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.545596:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5034 00000100:00100000:2.0:1788572581.555268:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.555408:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360707584 00000100:00100000:1.0:1788572581.555422:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360707584 00000100:00100000:1.0:1788572581.555496:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1097 00000100:00100000:1.0:1788572581.555506:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37414:x1875453360707584:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1788572581.556223:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37414:x1875453360707584:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 Request processed in 713us (962us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.556233:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1097 00000100:00100000:2.0:1788572581.574272:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.574590:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360707840 00000100:00100000:0.0:1788572581.574606:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360707840 00000100:00100000:0.0:1788572581.574690:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1098 00000100:00100000:0.0:1788572581.574703:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13e850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37414:x1875453360707840:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572581.575077:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13e850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37414:x1875453360707840:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 374us (819us total) trans 4294971127 rc 0/0 00000100:00100000:0.0:1788572581.575086:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1098 00000100:00100000:3.0:1788572581.579306:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.580964:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360708096 00000100:00100000:2.0:1788572581.580983:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360708096 00000100:00100000:3.0:1788572581.586280:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5035 00000100:00100000:3.0:1788572581.586320:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37414:x1875453360708096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1788572581.586510:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ff0a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37414:x1875453360708096:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 203us (7239us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.586519:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5035 00000100:00100000:2.0:1788572581.590771:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.592674:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360708352 00000100:00100000:0.0:1788572581.592695:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360708352 00000100:00100000:0.0:1788572581.592796:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5036 00000100:00100000:0.0:1788572581.592811:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37414:x1875453360708352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572581.592842:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572581.592871:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166785 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1ac3 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.592990:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.593051:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.593064:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.593103:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.593116:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 74 pid: 5837 timeout: 1029 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.593127:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382415488:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572581.593412:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c11ca3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572581.597196:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382415488:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572581.597325:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.603000:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.603729:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360708480 00000100:00100000:0.0:1788572581.603744:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360708480 00010000:00010000:0.0:1788572581.603811:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 75 pid: 5837 timeout: 1029 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.603866:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1201 00000100:00100000:0.0:1788572581.603878:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360708480:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572581.603897:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572581.603904:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 75 pid: 5837 timeout: 1029 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.603919:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 75 pid: 5837 timeout: 1029 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.603946:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646600/0x85ace67feb16677e lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1abc expref: 75 pid: 5837 timeout: 1029 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.603969:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572581.603976:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.603987:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.603997:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.604512:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572581.604726:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360708480:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 845us (1738us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572581.604736:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1201 00010000:00010000:2.0:1788572581.607106:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607128:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572581.607158:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb6:0x0]: returning size 0 00000004:00080000:2.0:1788572581.607623:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x61:0x0], index 1: 0 00010000:00010000:2.0:1788572581.607806:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607816:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607825:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607831:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572581.607837:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607845:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607866:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:2.0:1788572581.607869:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572581.607875:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607882:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16678c lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607962:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166785 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ac3 expref: 74 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607979:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572581.607982:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788572581.607987:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166785 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ac3 expref: 74 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.607997:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0200, rc 0) 00000100:00100000:2.0:1788572581.608139:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37414:x1875453360708352:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 15330us (17383us total) trans 4294971128 rc 301/301 00000100:00100000:2.0:1788572581.608149:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5036 00000100:00100000:2.0:1788572581.618359:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.618499:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360708736 00000100:00100000:0.0:1788572581.618514:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360708736 00000100:00100000:0.0:1788572581.618921:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1099 00000100:00100000:0.0:1788572581.618935:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37414:x1875453360708736:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572581.619501:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37414:x1875453360708736:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 566us (1153us total) trans 4294971129 rc 0/0 00000100:00100000:0.0:1788572581.619511:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1099 00000100:00100000:3.0:1788572581.626946:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.627596:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360708864 00000100:00100000:1.0:1788572581.627613:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360708864 00000100:00100000:1.0:1788572581.627734:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5037 00000100:00100000:1.0:1788572581.627749:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37414:x1875453360708864:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572581.627789:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572581.627820:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16679a lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1aca expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.627927:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1667a1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.627945:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1667a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.627959:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1667a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572581.628318:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb6:0x0]: returning size 0 00000004:00200000:1.0:1788572581.628324:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788572581.628356:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1667a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.628392:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646200/0x85ace67feb16679a lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aca expref: 76 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.628413:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1667a1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aca expref: 75 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.628424:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:1.0:1788572581.628427:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1788572581.628432:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647800, rc 0) 00000100:00100000:1.0:1788572581.628519:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37414:x1875453360708864:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 772us (1587us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.628596:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5037 00000100:00100000:2.0:1788572581.636249:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.636368:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360708992 00000100:00100000:3.0:1788572581.636381:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360708992 00000100:00100000:3.0:1788572581.636458:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1478 00000100:00100000:3.0:1788572581.636470:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360708992:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572581.636496:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572581.636535:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1ad1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572581.636573:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x61:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1788572581.636599:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=929s v=(0 0) 00010000:00010000:3.0:1788572581.636604:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572581.636616:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.636644:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 10 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.636662:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4a00, rc 0) 00000100:00100000:3.0:1788572581.636949:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360708992:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 479us (712us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.636958:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1478 00000100:00100000:2.0:1788572581.644893:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.645248:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360709248 00000100:00100000:1.0:1788572581.645262:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360709248 00000100:00100000:1.0:1788572581.645458:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5038 00000100:00100000:1.0:1788572581.645471:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37414:x1875453360709248:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1788572581.645637:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37414:x1875453360709248:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 169us (761us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.645646:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5038 00000100:00100000:2.0:1788572581.653521:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572581.653898:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360709376 00000100:00100000:1.0:1788572581.653912:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360709376 00000100:00100000:1.0:1788572581.653984:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1479 00000100:00100000:1.0:1788572581.653996:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:37414:x1875453360709376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572581.654129:0:23814:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572581.654156:0:23814:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 2/0,0 mode: --/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1ad8 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.654197:0:23814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.654209:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572581.654214:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572581.654220:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: --/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.654263:0:23814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.654348:0:23814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 11 pid: 6682 timeout: 1029 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572581.654366:0:23814:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:23814:1875453382415616:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572581.654472:0:23814:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f56cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572581.659419:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.659798:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360709504 00000100:00100000:2.0:1788572581.659813:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360709504 00010000:00010000:2.0:1788572581.659884:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 12 pid: 6682 timeout: 1029 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.659939:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1202 00000100:00100000:2.0:1788572581.659950:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360709504:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572581.659970:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572581.660000:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 12 pid: 6682 timeout: 1029 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.660053:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 12 pid: 6682 timeout: 1029 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.660598:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572581.660604:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572581.660617:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.660630:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.660682:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.660699:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c84ec000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382415744:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572581.660824:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c982de40 going to sleep for 11 seconds 00010000:00010000:1.0:1788572581.662673:0:23814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb4113a9/0x85ace67feb1667a8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad1 expref: 12 pid: 6682 timeout: 1029 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572581.662703:0:23814:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:23814:1875453382415616:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572581.662763:0:23814:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.662782:0:23814:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646200, rc 0) 00000100:00100000:1.0:1788572581.663262:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:37414:x1875453360709376:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 9264us (9753us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572581.663275:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1479 00000100:00100000:2.0:1788572581.680919:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c84ec000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5822:1875453382415744:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572581.681206:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572581.681862:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c4923100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360709504:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 21909us (22454us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572581.681879:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1202 00000100:00100000:2.0:1788572581.709839:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.710050:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360709632 00000100:00100000:3.0:1788572581.710188:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360709632 00002000:00100000:3.0:1788572581.710372:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x61:0x0] (0->1048575) req@ffff9627c84ec000 x1875453360709632/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:242/0 lens 488/0 e 0 to 0 dl 1788572592 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572581.710387:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ec000 with x1875453360709632 ext(0->1048575) 00002000:00010000:3.0:1788572581.710397:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c84ec000 00000100:00100000:3.0:1788572581.710447:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 385 00000100:00100000:3.0:1788572581.710462:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360709632:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572581.783668:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360709632:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 73185us (73821us total) trans 4294967680 rc 0/0 00000100:00100000:0.0:1788572581.783693:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 385 00002000:00100000:0.0:1788572581.783715:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x61:0x0] (0->1048575) req@ffff9627c84ec000 x1875453360709632/t4294967680(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:242/0 lens 488/448 e 0 to 0 dl 1788572592 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572581.783735:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84ec000 with x1875453360709632 ext(0->1048575) 00002000:00010000:0.0:1788572581.783743:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c84ec000 00000100:00100000:3.0:1788572581.788219:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.788471:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360709760 00000100:00100000:0.0:1788572581.788487:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360709760 00000100:00100000:0.0:1788572581.788570:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5039 00000100:00100000:0.0:1788572581.788581:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37414:x1875453360709760:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1788572581.858107:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37414:x1875453360709760:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 69509us (69886us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.858128:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5039 00000001:00080000:1.0:1788572581.859700:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971129 is committed 00000001:00080000:1.0:1788572581.859728:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971122 is committed 00000020:00080000:1.0:1788572581.859760:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627ca885480 refcount 2 committed 0 result 0 batchid 147 00000020:00080000:1.0:1788572581.859766:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ea8100 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627ca8850c0 00000020:00080000:1.0:1788572581.859770:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c2ea8a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c1e77000 00000001:00080000:1.0:1788572581.859827:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971123 is committed 00000020:00080000:3.0:1788572581.860425:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 147 committed batchid 146 00000020:00080000:3.0:1788572581.860941:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 146 00000100:00100000:2.0:1788572581.871774:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.872169:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360709888 00000100:00100000:0.0:1788572581.872183:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360709888 00000100:00100000:0.0:1788572581.872285:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1480 00000100:00100000:0.0:1788572581.872299:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360709888:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572581.919393:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360709888:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 47079us (47613us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.919412:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1480 00000001:00080000:3.0:1788572581.919478:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967680 is committed 00000001:00080000:3.0:1788572581.919494:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967679 is committed 00000100:00100000:2.0:1788572581.924890:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.925494:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360710144 00000100:00100000:0.0:1788572581.925510:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360710144 00000100:00100000:0.0:1788572581.925588:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5040 00000100:00100000:0.0:1788572581.925600:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37414:x1875453360710144:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572581.925682:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572581.925722:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647800/0x85ace67feb1667a1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aca expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.925760:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572581.925767:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572581.925807:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.925825:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.925843:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572581.926521:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c60c7eb4 reply state 00000000b7086004 transno 4294971130 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.926534:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572581.926669:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37414:x1875453360710144:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1070us (1794us total) trans 4294971130 rc 0/0 00000100:00100000:0.0:1788572581.926678:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5040 00000100:00100000:2.0:1788572581.934571:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.934722:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360710272 00000100:00100000:0.0:1788572581.934735:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360710272 00000100:00100000:0.0:1788572581.934823:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5041 00000100:00100000:0.0:1788572581.934835:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37414:x1875453360710272:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572581.934871:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572581.934900:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1adf expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.934988:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 62 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.935032:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 62 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.935071:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572581.935074:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572581.935079:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1400, rc 0) 00000100:00100000:0.0:1788572581.935154:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37414:x1875453360710272:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 321us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572581.935162:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5041 00010000:00010000:1.0:1788572581.936802:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.936822:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.936830:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.936836:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572581.936842:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.936850:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.936871:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:1.0:1788572581.936875:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572581.936880:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572581.936887:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb1667b6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.941642:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.941757:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360710400 00000100:00100000:0.0:1788572581.941771:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360710400 00000100:00100000:0.0:1788572581.941846:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1481 00000100:00100000:0.0:1788572581.941859:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360710400:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572581.942485:0:20911:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x61:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1788572581.942490:0:20911:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x61:0x0] updating lvb mtime from disk: 0 -> 1788572576 00002000:00010000:0.0:1788572581.942493:0:20911:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x61:0x0] updating lvb atime from disk: 0 -> 1788572576 00002000:00010000:0.0:1788572581.942495:0:20911:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x61:0x0] updating lvb ctime from disk: 0 -> 1788572576 00002000:00010000:0.0:1788572581.942497:0:20911:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x61:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:0.0:1788572581.942579:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360710400:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 721us (950us total) trans 4294967681 rc 0/0 00000100:00100000:0.0:1788572581.942589:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1481 00000100:00100000:2.0:1788572581.947339:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.947526:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360710656 00000100:00100000:0.0:1788572581.947540:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360710656 00000100:00100000:0.0:1788572581.947620:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5042 00000100:00100000:0.0:1788572581.947633:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37414:x1875453360710656:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1788572581.947731:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.947768:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 61 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.947779:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.947833:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 61 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.947849:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 61 pid: 9040 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.947863:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382415872:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572581.948139:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ffe82f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572581.952249:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382415872:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572581.952305:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.954889:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.955677:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360710784 00000100:00100000:3.0:1788572581.955693:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360710784 00010000:00010000:3.0:1788572581.955767:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 62 pid: 9040 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572581.955822:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1203 00000100:00100000:3.0:1788572581.955835:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360710784:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572581.955865:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572581.955873:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 62 pid: 9040 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.955888:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 62 pid: 9040 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.955917:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166770 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb6:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ab5 expref: 62 pid: 9040 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.955939:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:3.0:1788572581.955944:0:23057:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.955956:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.955980:0:23057:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.956473:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572581.956567:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360710784:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 732us (1691us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.956575:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1203 00010000:00010000:2.0:1788572581.957579:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957594:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957634:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957641:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957648:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957655:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572581.957660:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957669:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957685:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:2.0:1788572581.957689:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1788572581.957694:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.957701:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1667c4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.957797:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37414:x1875453360710656:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 10163us (10469us total) trans 0 rc -22/-22 00000100:00100000:2.0:1788572581.957808:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5042 00000100:00100000:2.0:1788572581.961871:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572581.962504:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360710912 00000100:00100000:3.0:1788572581.962519:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360710912 00000100:00100000:3.0:1788572581.962588:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1482 00000100:00100000:3.0:1788572581.962600:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360710912:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572581.962662:0:6682:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.962690:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.962700:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x61:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788572581.962705:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572581.962711:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 4/1,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.963362:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.963384:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 11 pid: 23814 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572581.963402:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36ddf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382416000:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572581.963515:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572581.970818:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572581.970952:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36ddf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6682:1875453382416000:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572581.971050:0:6682:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.971390:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360711040 00000100:00100000:0.0:1788572581.971400:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360711040 00010000:00010000:0.0:1788572581.971450:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.971501:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1204 00000100:00100000:0.0:1788572581.971513:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360711040:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572581.971535:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572581.971566:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.971581:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.971623:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d95f34e8/0x85ace67feb1667af lrc: 0/0,0 mode: --/PW res: [0x280000400:0x61:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ad8 expref: 12 pid: 23814 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.971644:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572581.971649:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572581.971656:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.971667:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.971704:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572581.971883:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360711040:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 370us (1080us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572581.971891:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1204 00010000:00010000:2.0:1788572581.972306:0:6682:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972319:0:6682:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972353:0:6682:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972363:0:6682:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972371:0:6682:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972379:0:6682:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572581.972385:0:6682:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972402:0:6682:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972415:0:6682:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.972424:0:6682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c877c2f7/0x85ace67feb1667cb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.972493:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360710912:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 9895us (10646us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572581.972502:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1482 00000100:00100000:2.0:1788572581.980454:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.980543:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360711296 00000100:00100000:0.0:1788572581.980556:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360711296 00000100:00100000:0.0:1788572581.980672:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5043 00000100:00100000:0.0:1788572581.980685:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37414:x1875453360711296:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572581.980805:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.980828:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.980843:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.981397:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.981414:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.981424:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.981467:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.981485:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 61 pid: 23058 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.981501:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382416128:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572581.981661:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ffe82f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572581.987470:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572581.987725:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382416128:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572581.987774:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.988615:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360711424 00000100:00100000:2.0:1788572581.988629:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360711424 00010000:00010000:2.0:1788572581.988689:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 62 pid: 23058 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572581.988861:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1205 00000100:00100000:2.0:1788572581.988875:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360711424:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572581.988897:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572581.988906:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 62 pid: 23058 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.988923:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 62 pid: 23058 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.988951:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1667bd lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1adf expref: 62 pid: 23058 timeout: 1030 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.988981:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:2.0:1788572581.988987:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.988999:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.989039:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572581.989089:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572581.989175:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360711424:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 303us (1719us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572581.989184:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1205 00010000:00010000:3.0:1788572581.989255:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572581.989281:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572581.989555:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971131 00000001:00080000:3.0:1788572581.990099:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971132 00000001:00080000:3.0:1788572581.990158:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971133 00000100:00100000:3.0:1788572581.990368:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382416256:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572581.990450:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572581.990494:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca9fad80 going to sleep for 11 seconds 00000100:00100000:3.0:1788572581.990532:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382416256 00000100:00100000:3.0:1788572581.990539:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382416256 00000100:00100000:3.0:1788572581.990599:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1788572581.990608:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382416256:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572581.990701:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382416256:12345-0@lo:1000:mdt00_002.0 Request processed in 93us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572581.990709:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00080000:3.0:1788572581.990732:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788572581.992151:0:5839:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967679 gen 2 00000004:00080000:3.0:1788572581.992168:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f596a680, transno 4294967676 00000004:00080000:3.0:1788572581.992197:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e7877b80, transno 4294967677 00000004:00080000:3.0:1788572581.992201:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627e7875c00, transno 4294967678 00000004:00080000:3.0:1788572581.992204:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627fb1e3800, transno 4294967679 00000100:00100000:3.0:1788572581.992214:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382416256:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1788572581.993693:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971134 00000004:00080000:1.0:1788572581.994231:0:7937:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 1 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:3.0:1788572581.995201:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382416384:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572581.995383:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572581.995439:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077780 going to sleep for 11 seconds 00000100:00100000:1.0:1788572581.995628:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382416384 00000100:00100000:1.0:1788572581.995638:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382416384 00000100:00100000:1.0:1788572581.995713:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:1.0:1788572581.995725:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382416384:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1788572581.996867:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382416384:12345-0@lo:1000:mdt00_002.0 Request processed in 1138us (1478us total) trans 4294967682 rc 0/0 00000100:00100000:1.0:1788572581.996883:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1788572581.997100:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c1bcdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382416384:0@lo:1000:mdt00_002.0 00000001:00080000:0.0:1788572581.997530:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971135 00010000:00010000:0.0:1788572581.997590:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997604:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997613:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997620:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572581.997625:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997636:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997658:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9626c4b10a00) 00010000:00010000:0.0:1788572581.997662:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572581.997669:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997676:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667d9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb6:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997721:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997728:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997735:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997871:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572581.997875:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997885:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997895:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10600) 00010000:00010000:0.0:1788572581.997898:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572581.997909:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 242 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997923:0:5839:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 242 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.997934:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 242 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572581.998197:0:5839:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9600/0x85ace67feb166700 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1a99 expref: 242 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572581.998216:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382416512:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572581.998520:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ffe82f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572582.006289:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.006455:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360711552 00000100:00100000:1.0:1788572582.006468:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360711552 00000100:00080000:3.0:1788572582.006597:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788572582.006642:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1100 00000100:00100000:1.0:1788572582.006655:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37412:x1875453360711552:12345-192.168.202.40@tcp:49:md5sum.0 00000100:00100000:1.0:1788572582.006922:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37412:x1875453360711552:12345-192.168.202.40@tcp:49:md5sum.0 Request processed in 268us (664us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572582.006931:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1100 00000100:00100000:0.0:1788572582.009467:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbf100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382416512:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572582.009520:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009536:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1666f9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009590:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009610:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009625:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009639:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009645:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009652:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009658:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.009663:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009671:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009686:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10600) 00010000:00010000:0.0:1788572582.009690:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.009694:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009700:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667e0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009712:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009718:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009723:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009728:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.009730:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009735:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009742:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb1:0x0].0x0 (ffff9626c4b10600) 00010000:00010000:0.0:1788572582.009744:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572582.009747:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.009752:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1a00/0x85ace67feb1667d2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572582.010379:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37414:x1875453360711296:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 29692us (29934us total) trans 4294971136 rc 0/0 00000100:00100000:0.0:1788572582.010393:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5043 00000020:00080000:0.0:1788572582.010402:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:3.0:1788572582.011065:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572582.011073:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572582.011725:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 8 00000040:00080000:3.0:1788572582.011809:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1788572582.014420:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572582.017879:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572582.018372:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360711680 00000100:00100000:0.0:1788572582.018387:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360711680 00000100:00100000:2.0:1788572582.018882:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.019052:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360711936 00000100:00100000:1.0:1788572582.019059:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360711936 00000100:00100000:0.0:1788572582.019238:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1483 00000100:00100000:0.0:1788572582.019253:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a73dcc9b-61e8-4b8a-971c-1141da788d66+83:37412:x1875453360711680:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572582.019869:0:23814:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572582.019903:0:23814:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009b75b8bf/0x85ace67feb1667e7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1ae6 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572582.019941:0:23814:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x61:0x0] initial LVB size: 1048576, mtime: 0x6a9b73a0, atime: 0x6a9b73a0, ctime: 0x6a9b73a0, blocks: 0x803 00010000:00010000:0.0:1788572582.019970:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=930s v=(0 0) 00010000:00010000:0.0:1788572582.019976:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x61:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572582.019989:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009b75b8bf/0x85ace67feb1667e7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ae6 expref: 84 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.020219:0:23814:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009b75b8bf/0x85ace67feb1667e7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x61:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ae6 expref: 84 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.020236:0:23814:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1a00, rc 0) 00000100:00100000:0.0:1788572582.020352:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:37412:x1875453360711680:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 1102us (2488us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572582.020361:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1483 00000100:00080000:3.0:1788572582.022638:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788572582.022654:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788572582.023653:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1101 00000100:00100000:2.0:1788572582.023671:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37414:x1875453360711936:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572582.024624:0:5840:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xb6:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1788572582.024881:0:5840:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971137 00010000:00010000:2.0:1788572582.024947:0:5840:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572582.024969:0:5840:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572582.024988:0:5840:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572582.024998:0:5840:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572582.025180:0:5840:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572582.025188:0:5840:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572582.025213:0:5840:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788572582.025397:0:7615:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572582.025407:0:7615:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572582.025414:0:7615:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9e00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572582.025419:0:7615:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572582.025430:0:7615:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572582.025445:0:7615:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb6:0x0].0x0 (ffff9627e3306f00) 00010000:00010000:3.0:1788572582.025449:0:7615:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572582.025454:0:7615:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 1/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572582.025460:0:7615:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb1667ee lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5840 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572582.025479:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572582.025481:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572582.025483:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572582.025627:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37414:x1875453360711936:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1958us (6745us total) trans 4294971138 rc 0/0 00000100:00100000:2.0:1788572582.025639:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1101 00000020:00080000:2.0:1788572582.025646:0:5840:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788572582.026991:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.027407:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360712064 00000100:00100000:1.0:1788572582.027420:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360712064 00002000:00100000:1.0:1788572582.027475:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x61:0x0] (0->32767) req@ffff962801f94000 x1875453360712064/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:243/0 lens 488/0 e 0 to 0 dl 1788572593 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572582.027495:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f94000 with x1875453360712064 ext(0->32767) 00002000:00010000:1.0:1788572582.027501:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f94000 00000100:00100000:1.0:1788572582.027577:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 386 00000100:00100000:1.0:1788572582.027587:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360712064:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:1.0:1788572582.034126:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360712064:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 5132us (5733us total) trans 0 rc 32768/32768 00000100:00100000:1.0:1788572582.034158:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 386 00002000:00100000:1.0:1788572582.034180:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x61:0x0] (0->32767) req@ffff962801f94000 x1875453360712064/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:243/0 lens 488/440 e 0 to 0 dl 1788572593 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572582.034192:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f94000 with x1875453360712064 ext(0->32767) 00002000:00010000:1.0:1788572582.034201:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f94000 00000100:00100000:2.0:1788572582.042997:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.043375:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360712192 00000100:00100000:1.0:1788572582.043391:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360712192 00002000:00100000:1.0:1788572582.043467:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x61:0x0] (32768->1048575) req@ffff962801f96d80 x1875453360712192/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:243/0 lens 488/0 e 0 to 0 dl 1788572593 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572582.043480:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f96d80 with x1875453360712192 ext(32768->1048575) 00002000:00010000:1.0:1788572582.043487:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f96d80 00000100:00100000:1.0:1788572582.043530:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 387 00000100:00100000:1.0:1788572582.043542:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360712192:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:1.0:1788572582.069479:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:2349:x1875453360712192:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 25921us (26480us total) trans 0 rc 1015808/1015808 00000100:00100000:1.0:1788572582.069498:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 387 00002000:00100000:1.0:1788572582.069516:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x61:0x0] (32768->1048575) req@ffff962801f96d80 x1875453360712192/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:243/0 lens 488/440 e 0 to 0 dl 1788572593 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572582.069528:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f96d80 with x1875453360712192 ext(32768->1048575) 00002000:00010000:1.0:1788572582.069536:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff962801f96d80 00000100:00100000:3.0:1788572582.084375:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.084644:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360712448 00000100:00100000:1.0:1788572582.084662:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360712448 00000100:00100000:1.0:1788572582.084803:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1102 00000100:00100000:1.0:1788572582.084818:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+243:37412:x1875453360712448:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:1.0:1788572582.085248:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37412:x1875453360712448:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 432us (892us total) trans 4294971139 rc 0/0 00000100:00100000:1.0:1788572582.085589:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1102 00000100:00100000:2.0:1788572582.116382:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572582.116499:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360712704 00000100:00100000:0.0:1788572582.116512:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360712704 00000100:00100000:0.0:1788572582.116588:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5044 00000100:00100000:0.0:1788572582.116600:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+244:37416:x1875453360712704:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572582.116635:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572582.116661:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667f5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1aed expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.116781:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.116803:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.116818:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.116836:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.116843:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.116849:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117496:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb16680a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117509:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb16680a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117518:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb16680a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572582.117532:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb2:0x0]: returning size 0 00010000:00010000:0.0:1788572582.117657:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117665:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117672:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117679:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.117684:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117693:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117709:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572582.117713:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.117720:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.117728:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1667fc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119313:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119324:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119332:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119339:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.119344:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119354:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119375:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xd5c90681 (ffff9627c5160500) 00010000:00010000:0.0:1788572582.119378:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.119383:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119390:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166803 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xd5c90681 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119480:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1667f5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aed expref: 247 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119497:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb16680a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aed expref: 246 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.119505:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160900) 00010000:00010000:0.0:1788572582.119508:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572582.119510:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1e00, rc 0) 00000100:00100000:0.0:1788572582.119701:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37416:x1875453360712704:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 3100us (3332us total) trans 4294971140 rc 0/0 00000100:00100000:0.0:1788572582.119712:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5044 00000100:00100000:2.0:1788572582.142515:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572582.142589:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572582.142752:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713088 00000100:00100000:0.0:1788572582.142766:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713088 00000100:00100000:0.0:1788572582.142857:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 435 00000100:00100000:0.0:1788572582.142870:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c97cd450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:2350:x1875453360713088:12345-192.168.202.40@tcp:63:ll_sa.0 00010000:00010000:0.0:1788572582.142968:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572582.143000:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb166818 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1af4 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143134:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143155:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143172:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143189:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143197:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143204:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143554:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143564:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143573:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572582.143599:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00000004:00200000:0.0:1788572582.143604:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572582.143633:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143652:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143663:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143671:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143678:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143685:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.143691:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143700:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143716:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572582.143719:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.143725:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143732:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb16681f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143748:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143754:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143762:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143767:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.143769:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143776:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143782:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9627c5160500) 00010000:00010000:0.0:1788572582.143784:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.143787:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143792:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166826 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143831:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb166818 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1af4 expref: 248 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143846:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1af4 expref: 247 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143853:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627c5160000) 00010000:00010000:0.0:1788572582.143855:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572582.143858:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0400, rc 0) 00010000:00010000:0.0:1788572582.143865:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572582.143873:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166834 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1afb expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143899:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143908:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143915:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143927:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143933:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.143939:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144131:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144140:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144147:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572582.144162:0:18263:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00000004:00200000:0.0:1788572582.144167:0:18263:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572582.144178:0:18263:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144191:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144198:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144204:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144210:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144216:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.144219:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144226:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144234:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572582.144236:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.144239:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144245:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb16683b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144255:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144273:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144280:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144285:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.144288:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144294:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144302:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627c5160d00) 00010000:00010000:0.0:1788572582.144304:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.144307:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144314:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166842 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144337:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb166834 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1afb expref: 249 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144349:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1afb expref: 248 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144356:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:0.0:1788572582.144359:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572582.144361:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0e00, rc 0) 00010000:00010000:0.0:1788572582.144367:0:18263:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572582.144375:0:18263:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166850 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b02 expref: -99 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144401:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144410:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144418:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144431:0:18263:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144437:0:18263:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144444:0:18263:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144525:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144532:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144537:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144543:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.144546:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144557:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144566:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572582.144568:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.144571:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144576:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166857 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144585:0:18263:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144591:0:18263:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144596:0:18263:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144600:0:18263:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572582.144603:0:18263:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144609:0:18263:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144615:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xb3dc5f41 (ffff9627fb58df00) 00010000:00010000:0.0:1788572582.144618:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572582.144620:0:18263:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144626:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb16685e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xb3dc5f41 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18263 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144648:0:18263:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166850 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b02 expref: 249 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144658:0:18263:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572582.144660:0:18263:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788572582.144664:0:18263:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb166850 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b02 expref: 249 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.144672:0:18263:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0600, rc -2) 00000100:00100000:0.0:1788572582.144765:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c97cd450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2350:x1875453360713088:12345-192.168.202.40@tcp:63:ll_sa.0 Request processed in 1895us (2264us total) trans 0 rc -2/-2 00000100:00100000:0.0:1788572582.144775:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 435 00000100:00100000:1.0:1788572582.147663:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360712960 00000100:00100000:1.0:1788572582.147685:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360712960 00000100:00100000:1.0:1788572582.148140:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 382 00000100:00100000:1.0:1788572582.148158:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37416:x1875453360712960:12345-192.168.202.40@tcp:103:md5sum.0 00010000:00010000:1.0:1788572582.148180:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572582.148213:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb16680a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1aed expref: 248 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572582.148241:0:23057:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627c5160900) 00010000:00010000:1.0:1788572582.148257:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572582.148402:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37416:x1875453360712960:12345-192.168.202.40@tcp:103:md5sum.0 Request processed in 248us (5812us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572582.149513:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 382 00000100:00100000:3.0:1788572582.150642:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.150742:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713216 00000100:00100000:1.0:1788572582.150754:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713216 00000100:00100000:1.0:1788572582.150818:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5045 00000100:00100000:1.0:1788572582.150830:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37416:x1875453360713216:12345-192.168.202.40@tcp:58:md5sum.0 00010000:00010000:1.0:1788572582.151225:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572582.151249:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572582.151347:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1788572582.151802:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].176, 136 off31920 00000100:00100000:1.0:1788572582.151934:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37416:x1875453360713216:12345-192.168.202.40@tcp:58:md5sum.0 Request processed in 1103us (1296us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572582.151944:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5045 00000100:00100000:2.0:1788572582.156894:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572582.157469:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713344 00000100:00100000:1.0:1788572582.157486:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713344 00000100:00100000:0.0:1788572582.157662:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5046 00000100:00100000:0.0:1788572582.157675:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37416:x1875453360713344:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572582.157705:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572582.157865:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b2c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.157945:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.158167:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 4/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.158198:0:5837:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.158214:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.158230:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:0.0:1788572582.158233:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572582.158238:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572582.158246:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0600, rc 0) 00000100:00100000:0.0:1788572582.158473:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f95880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37416:x1875453360713344:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 798us (1591us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572582.158483:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5046 00000040:00080000:3.0:1788572583.034411:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572583.034423:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572583.034471:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 177 count 9 00000040:00080000:3.0:1788572583.034482:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572583.034654:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ca8c9c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382416640:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572583.034808:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077000 going to sleep for 11 seconds 00010000:00010000:3.0:1788572583.038657:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572583.056862:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ca8c9c00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382416640:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572583.058750:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.058847:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713472 00000100:00100000:2.0:1788572583.058858:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713472 00000100:00100000:2.0:1788572583.058925:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5047 00000100:00100000:2.0:1788572583.058938:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37420:x1875453360713472:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1788572583.058989:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb2:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1788572583.059164:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37420:x1875453360713472:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 226us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572583.059172:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5047 00000100:00100000:2.0:1788572583.069695:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.069821:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713600 00000100:00100000:0.0:1788572583.069836:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713600 00000100:00100000:0.0:1788572583.069917:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5048 00000100:00100000:0.0:1788572583.069930:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37420:x1875453360713600:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.070077:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070100:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070116:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070138:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070146:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070154:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070182:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070190:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070197:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572583.070580:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb2:0x0]: returning size 0 00000004:00200000:0.0:1788572583.070586:0:22536:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572583.070604:0:22536:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070619:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070625:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070632:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070642:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572583.070646:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070654:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070669:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572583.070672:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572583.070678:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070684:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166873 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070700:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070706:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070712:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070716:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572583.070719:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070725:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070736:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x2ff0a106 (ffff9627fb58df00) 00010000:00010000:0.0:1788572583.070739:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572583.070741:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070748:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb16687a lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2ff0a106 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070765:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070771:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070777:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070782:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572583.070785:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070790:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070798:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:0.0:1788572583.070800:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572583.070804:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.070809:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166881 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.070987:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37420:x1875453360713600:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 1056us (1307us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.070997:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5048 00000100:00100000:3.0:1788572583.085453:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.085526:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713856 00000100:00100000:0.0:1788572583.085540:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713856 00000100:00100000:0.0:1788572583.085681:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5049 00000100:00100000:0.0:1788572583.085696:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37420:x1875453360713856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.085737:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572583.085777:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166888 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b33 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.085942:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.085963:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.085987:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572583.086000:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb2:0x0]: returning size 0 00010000:00010000:0.0:1788572583.086190:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086199:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086207:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086237:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572583.086241:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086249:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086264:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:0.0:1788572583.086267:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572583.086273:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086280:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16688f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086334:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166888 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b33 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086344:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572583.086347:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572583.086351:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166888 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b33 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.086361:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1800, rc 0) 00000100:00100000:0.0:1788572583.086438:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37420:x1875453360713856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 744us (1011us total) trans 4294971141 rc 301/301 00000100:00100000:0.0:1788572583.086447:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5049 00000100:00100000:3.0:1788572583.091045:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.091265:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360713984 00000100:00100000:1.0:1788572583.091280:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360713984 00000100:00100000:0.0:1788572583.091417:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5050 00000100:00100000:0.0:1788572583.091433:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37420:x1875453360713984:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572583.091487:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb2:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572583.091637:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37420:x1875453360713984:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 207us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.091646:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5050 00000100:00100000:1.0:1788572583.097275:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572583.097573:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360714240 00000100:00100000:3.0:1788572583.097587:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360714240 00000100:00100000:3.0:1788572583.097722:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5051 00000100:00100000:3.0:1788572583.097735:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37420:x1875453360714240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572583.097766:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572583.097797:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb16689d lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b3a expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.098935:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1668a4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.098957:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1668a4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.099871:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1668a4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572583.099896:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb7:0x0]: returning size 0 00010000:00010000:3.0:1788572583.100639:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb16689d lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b3a expref: 71 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.100670:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1668a4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b3a expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.100680:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:3.0:1788572583.100684:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572583.100688:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4a00, rc 0) 00000100:00100000:3.0:1788572583.100988:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37420:x1875453360714240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 3244us (3917us total) trans 4294971142 rc 0/0 00000100:00100000:3.0:1788572583.100999:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5051 00000100:00100000:3.0:1788572583.104332:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.104549:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360714496 00000100:00100000:2.0:1788572583.104563:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360714496 00000100:00100000:2.0:1788572583.104638:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5052 00000100:00100000:2.0:1788572583.104651:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37420:x1875453360714496:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572583.104728:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572583.104763:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb1668a4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b3a expref: 70 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.104797:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9626c4b10800) 00010000:00010000:2.0:1788572583.104821:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788572583.104853:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.104871:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.104884:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.104901:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.104908:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.104915:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106579:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106598:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106606:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106613:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffe1600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572583.106618:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106627:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106645:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572583.106647:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572583.106653:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106659:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe1600/0x85ace67feb1668b2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106673:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106680:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106685:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106691:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffe0600 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572583.106694:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106699:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106707:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xa3649622 (ffff9627e3306700) 00010000:00010000:2.0:1788572583.106710:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572583.106713:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.106718:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0600/0x85ace67feb1668b9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xa3649622 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572583.107543:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37420:x1875453360714496:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2881us (3220us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572583.107556:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5052 00000100:00100000:2.0:1788572583.110658:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.110763:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360714624 00000100:00100000:1.0:1788572583.110778:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360714624 00000100:00100000:1.0:1788572583.111048:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5053 00000100:00100000:1.0:1788572583.111061:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37420:x1875453360714624:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572583.111104:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572583.111138:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1668c0 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b41 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.111185:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.111202:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.111216:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.111372:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb1668c0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 71 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.111394:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 70 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.111403:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572583.111405:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1788572583.111409:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb646800, rc 0) 00000100:00100000:1.0:1788572583.111487:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37420:x1875453360714624:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 429us (845us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572583.111494:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5053 00000100:00100000:1.0:1788572583.115752:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.116058:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360714880 00000100:00100000:0.0:1788572583.116089:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360714880 00000100:00100000:0.0:1788572583.116178:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5054 00000100:00100000:0.0:1788572583.116191:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37420:x1875453360714880:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.116330:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.116960:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.116978:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572583.118259:0:22536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000065467e21 reply state 000000004ab57c60 transno 4294971143 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.118279:0:22536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572583.118640:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37420:x1875453360714880:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 2445us (2898us total) trans 4294971143 rc 0/0 00000100:00100000:0.0:1788572583.118652:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5054 00000100:00100000:3.0:1788572583.130452:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.130598:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360715008 00000100:00100000:0.0:1788572583.130613:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360715008 00000100:00100000:0.0:1788572583.130692:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5055 00000100:00100000:0.0:1788572583.130705:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37420:x1875453360715008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.130741:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572583.130772:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b48 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.130866:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 73 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.130885:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 73 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.130897:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572583.130900:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572583.130905:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1200, rc 0) 00000100:00100000:0.0:1788572583.130989:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360715008:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 286us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.130998:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5055 00010000:00010000:1.0:1788572583.131724:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.131737:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.131745:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.131752:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572583.131759:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.131768:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.131783:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572583.131785:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572583.131790:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.131796:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb1668ce lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572583.152209:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.152350:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360715136 00000100:00100000:1.0:1788572583.152364:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360715136 00000100:00100000:1.0:1788572583.152472:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1103 00000100:00100000:1.0:1788572583.152485:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13f850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37420:x1875453360715136:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1788572583.152774:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13f850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37420:x1875453360715136:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 Request processed in 291us (578us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572583.152794:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1103 00000100:00100000:2.0:1788572583.170944:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.171111:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360715392 00000100:00100000:1.0:1788572583.171124:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360715392 00000100:00100000:1.0:1788572583.171213:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1104 00000100:00100000:1.0:1788572583.171238:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37420:x1875453360715392:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572583.171602:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360715392:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 378us (674us total) trans 4294971144 rc 0/0 00000100:00100000:1.0:1788572583.171611:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1104 00000100:00100000:3.0:1788572583.182610:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.182776:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360715648 00000100:00100000:0.0:1788572583.182790:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360715648 00000100:00100000:0.0:1788572583.182875:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5056 00000100:00100000:0.0:1788572583.182888:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360715648:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572583.183103:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360715648:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 217us (509us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.183112:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5056 00000100:00100000:3.0:1788572583.206622:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.206947:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360715904 00000100:00100000:1.0:1788572583.206962:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360715904 00000100:00100000:0.0:1788572583.213409:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5057 00000100:00100000:0.0:1788572583.213448:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360715904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.213483:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572583.213514:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1668dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b4f expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.213643:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.213661:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.213683:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.213734:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 74 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.213758:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 74 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.213774:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382416768:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572583.213921:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572583.225567:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382416768:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572583.225613:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1788572583.226384:0:6873:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 931295377523ns, next at 932295374534ns, rc = 0 00000100:00100000:1.0:1788572583.226449:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788572583.226474:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382416896:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788572583.226554:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572583.226712:0:6684:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382416896 00000100:00100000:2.0:1788572583.226721:0:6684:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382416896 00000100:00100000:2.0:1788572583.226803:0:6684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1788572583.226816:0:6684:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382416896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788572583.226860:0:6684:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 198901760 pending 0 free 3854565376 avail 3852468224 00000100:00100000:2.0:1788572583.226925:0:6684:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3303:x1875453382416896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 113us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572583.226934:0:6684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:1.0:1788572583.227058:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382416896:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1788572583.227124:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572583.229562:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.229799:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360716032 00000100:00100000:0.0:1788572583.229813:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360716032 00010000:00010000:0.0:1788572583.229872:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 75 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.229926:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1206 00000100:00100000:0.0:1788572583.229939:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360716032:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572583.229957:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572583.229966:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 75 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.229981:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 75 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.230056:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668d5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b48 expref: 75 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.230080:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572583.230086:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.230099:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.230107:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.230139:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572583.230205:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360716032:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 269us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.230213:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1206 00010000:00010000:1.0:1788572583.230470:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.230479:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572583.230505:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb7:0x0]: returning size 0 00000004:00080000:1.0:1788572583.230963:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x62:0x0], index 0: 0 00010000:00010000:1.0:1788572583.231186:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231197:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231204:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231211:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572583.231216:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231237:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231252:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572583.231256:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572583.231261:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231267:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1668e3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231340:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1668dc lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b4f expref: 74 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231351:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:1.0:1788572583.231353:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788572583.231358:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb1668dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b4f expref: 74 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.231368:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0c00, rc 0) 00000100:00100000:1.0:1788572583.231421:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360715904:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 17996us (24816us total) trans 4294971145 rc 301/301 00000100:00100000:1.0:1788572583.231430:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5057 00000100:00100000:2.0:1788572583.236590:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.236676:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360716288 00000100:00100000:0.0:1788572583.236699:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360716288 00000100:00100000:0.0:1788572583.236765:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1105 00000100:00100000:0.0:1788572583.236783:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37420:x1875453360716288:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1788572583.237292:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37420:x1875453360716288:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 510us (713us total) trans 4294971146 rc 0/0 00000100:00100000:0.0:1788572583.237302:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1105 00000100:00100000:3.0:1788572583.240899:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.243671:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360716416 00000100:00100000:0.0:1788572583.243714:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360716416 00000100:00100000:0.0:1788572583.243819:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5058 00000100:00100000:0.0:1788572583.243832:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37420:x1875453360716416:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.243867:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572583.243895:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668f1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b56 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.243998:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb1668f8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.244070:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb1668f8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.244085:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb1668f8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572583.244132:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb7:0x0]: returning size 0 00000004:00200000:0.0:1788572583.244137:0:5839:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572583.244161:0:5839:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb1668f8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.244195:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1668f1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b56 expref: 76 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.244252:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb1668f8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b56 expref: 75 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.244262:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572583.244265:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572583.244269:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0200, rc 0) 00000100:00100000:0.0:1788572583.244367:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37420:x1875453360716416:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 537us (3483us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.244375:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5058 00000100:00100000:3.0:1788572583.257738:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.258063:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360716544 00000100:00100000:0.0:1788572583.258082:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360716544 00000100:00100000:0.0:1788572583.258168:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1484 00000100:00100000:0.0:1788572583.258180:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360716544:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.258204:0:10416:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572583.258242:0:10416:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b5d expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572583.258279:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x62:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572583.258318:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=931s v=(0 0) 00010000:00010000:0.0:1788572583.258324:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00000100:00080000:1.0:1788572583.258493:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1788572583.258536:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 12 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.258563:0:10416:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 12 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.258586:0:10416:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1200, rc 0) 00000100:00100000:0.0:1788572583.258929:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360716544:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 748us (1205us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.258938:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1484 00000100:00100000:2.0:1788572583.273734:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.274501:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360716800 00000100:00100000:0.0:1788572583.274517:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360716800 00000100:00100000:2.0:1788572583.274844:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5059 00000100:00100000:2.0:1788572583.274859:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37420:x1875453360716800:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572583.275204:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37420:x1875453360716800:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 346us (1484us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572583.275213:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5059 00000100:00100000:2.0:1788572583.283621:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.284490:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360716928 00000100:00100000:1.0:1788572583.284508:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360716928 00000100:00100000:1.0:1788572583.284596:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1485 00000100:00100000:1.0:1788572583.284608:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:37420:x1875453360716928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572583.284631:0:13272:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572583.284658:0:13272:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b64 expref: -99 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.284714:0:13272:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.284726:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572583.284732:0:13272:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572583.285213:0:13272:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.285927:0:13272:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.285952:0:13272:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572583.285969:0:13272:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382417024:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572583.286997:0:13272:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff548780 going to sleep for 11 seconds 00000100:00100000:2.0:1788572583.303068:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.303374:0:13272:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13272:1875453382417024:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572583.303419:0:13272:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 13 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.303446:0:13272:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647c00, rc 0) 00000100:00100000:1.0:1788572583.303517:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:37420:x1875453360716928:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 18911us (19911us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572583.303528:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1485 00000100:00100000:0.0:1788572583.304164:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717056 00000100:00100000:0.0:1788572583.304176:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717056 00010000:00010000:0.0:1788572583.304236:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.304282:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1207 00000100:00100000:0.0:1788572583.304294:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360717056:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572583.304310:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572583.304350:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.304366:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.304400:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5127d53/0x85ace67feb1668ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b5d expref: 13 pid: 10416 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.304420:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572583.304426:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572583.304439:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.304450:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.304498:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.304515:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382417152:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572583.304597:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572583.311088:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382417152:192.168.202.40@tcp:105: 00010000:00010000:1.0:1788572583.311128:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572583.311208:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360717056:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 6915us (8153us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572583.311234:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1207 00000100:00100000:2.0:1788572583.344529:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.344662:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717184 00000100:00100000:0.0:1788572583.344675:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717184 00002000:00100000:0.0:1788572583.344747:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x62:0x0] (0->1048575) req@ffff9627fb64aa00 x1875453360717184/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:244/0 lens 488/0 e 0 to 0 dl 1788572594 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572583.344762:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb64aa00 with x1875453360717184 ext(0->1048575) 00002000:00010000:0.0:1788572583.344769:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb64aa00 00000100:00100000:0.0:1788572583.344819:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 388 00000100:00100000:0.0:1788572583.344831:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360717184:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1788572583.415923:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360717184:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 71076us (71390us total) trans 4294967682 rc 0/0 00000100:00100000:0.0:1788572583.415944:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 388 00002000:00100000:0.0:1788572583.416918:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x62:0x0] (0->1048575) req@ffff9627fb64aa00 x1875453360717184/t4294967682(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:244/0 lens 488/448 e 0 to 0 dl 1788572594 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572583.416939:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627fb64aa00 with x1875453360717184 ext(0->1048575) 00002000:00010000:0.0:1788572583.416947:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627fb64aa00 00000100:00100000:0.0:1788572583.428623:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.428931:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717312 00000100:00100000:2.0:1788572583.428946:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717312 00000100:00100000:2.0:1788572583.429081:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5060 00000100:00100000:2.0:1788572583.429095:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37420:x1875453360717312:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000001:00080000:1.0:1788572583.469155:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971146 is committed 00000001:00080000:1.0:1788572583.469182:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971140 is committed 00000004:00080000:1.0:1788572583.469224:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 50 committed 00000020:00080000:1.0:1788572583.469260:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627d13eb000 refcount 2 committed 0 result 0 batchid 148 00000020:00080000:1.0:1788572583.469267:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c4ffbc00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627d13ebc00 00000020:00080000:1.0:1788572583.469271:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c4ffb200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627ca982c00 00000020:00080000:1.0:1788572583.469294:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 147 batchid updated 00000020:00080000:3.0:1788572583.469509:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 148 committed batchid 147 00000020:00080000:3.0:1788572583.469771:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 147 00000100:00100000:3.0:1788572583.469784:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788572583.469812:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382417280:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1788572583.469899:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.470166:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788572583.470177:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382417408:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1788572583.470219:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.471399:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37420:x1875453360717312:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 42298us (42781us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.471415:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5060 00000100:00100000:1.0:1788572583.473772:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382417280 00000100:00100000:1.0:1788572583.473799:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382417280 00000100:00100000:0.0:1788572583.474540:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382417408 00000100:00100000:0.0:1788572583.474549:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382417408 00000100:00100000:0.0:1788572583.474593:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1486 00000100:00100000:0.0:1788572583.474607:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+10:3302:x1875453382417280:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1788572583.474653:0:10416:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:96 count 1 00002000:00010000:0.0:1788572583.474714:0:10416:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x60:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572583.474724:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.474744:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=931s v=(0 0) 00010000:00010000:0.0:1788572583.474749:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572583.474757:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.474775:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.474786:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.474795:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.474831:0:10416:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572583.475789:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3302:x1875453382417280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1183us (5886us total) trans 4294967683 rc 0/0 00000100:00100000:0.0:1788572583.475800:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1486 00000100:00100000:0.0:1788572583.475833:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1487 00000100:00100000:0.0:1788572583.475846:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+10:3305:x1875453382417408:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1788572583.475867:0:10416:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:94 count 1 00010000:00010000:0.0:1788572583.475888:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.475910:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 72 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.475921:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.475930:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=7s v=(0 1) 00010000:00010000:0.0:1788572583.475934:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572583.475940:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.476283:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572583.476293:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.476312:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572583.476317:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.476329:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.476358:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 1/0,0 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.476366:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb16690d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.476399:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572583.476402:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572583.476404:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572583.476511:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 13 pid: 20911 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.476531:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 13 pid: 20911 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.476550:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382417536:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572583.477517:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 72 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.477536:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 72 pid: 6679 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.477548:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382417664:192.168.202.40@tcp:104: 00000004:00080000:3.0:1788572583.477854:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f56f1500/2, rc 0, transno 4294967683 00000100:00100000:3.0:1788572583.477864:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3302:1875453382417280:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1788572583.478070:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.478787:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717440 00000100:00100000:1.0:1788572583.478799:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717440 00000100:00100000:1.0:1788572583.478884:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1488 00000100:00100000:1.0:1788572583.478897:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2349:x1875453360717440:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:0.0:1788572583.481750:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e45cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572583.481802:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382417536:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572583.494844:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382417664:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572583.494903:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572583.495479:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572583.495521:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717568 00000100:00100000:3.0:1788572583.495530:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717568 00010000:00010000:3.0:1788572583.495597:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572583.495636:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1208 00000100:00100000:3.0:1788572583.495646:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2350:x1875453360717568:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572583.495664:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572583.495672:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.495696:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.495725:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22b7312/0x85ace67feb166015 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400030020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1831 expref: 73 pid: 6679 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.495746:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572583.495807:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:2350:x1875453360717568:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 161us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.495815:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1208 00000100:00080000:3.0:1788572583.495843:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788572583.502800:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.503306:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717696 00000100:00100000:1.0:1788572583.503322:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717696 00010000:00010000:1.0:1788572583.503377:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 15 pid: 20911 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572583.503427:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1209 00000100:00100000:1.0:1788572583.503438:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+15:2350:x1875453360717696:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572583.503457:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572583.503464:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 15 pid: 20911 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.503486:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 15 pid: 20911 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.503514:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003de67b4f/0x85ace67feb166085 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1854 expref: 15 pid: 20911 timeout: 1031 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.503534:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572583.503539:0:5823:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572583.503547:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.503558:0:5823:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.503591:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572583.503649:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+14:2350:x1875453360717696:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 213us (862us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572583.503657:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1209 00010000:00010000:0.0:1788572583.508198:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.508234:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.508250:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.508259:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.508775:0:10416:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572583.509720:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+11:3305:x1875453382417408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 33870us (39503us total) trans 4294967684 rc 0/0 00000100:00100000:0.0:1788572583.509736:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1487 00000004:00080000:3.0:1788572583.512229:0:3305:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9627f56f0000/2, rc 0, transno 4294967684 00000100:00100000:3.0:1788572583.512256:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382417408:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1788572583.516421:0:7616:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572583.516442:0:7616:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.516459:0:7616:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572583.516466:0:7616:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.516485:0:7616:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.516514:0:7616:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.516523:0:7616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f30003d0/0x85ace67feb166914 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.516552:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572583.516554:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572583.516556:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1788572583.543575:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967683 is committed 00000001:00080000:2.0:1788572583.543597:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967682 is committed 00000100:00100000:0.0:1788572583.543892:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360717440:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 64989us (65814us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.543907:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1488 00000100:00100000:3.0:1788572583.547484:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.547602:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360717952 00000100:00100000:0.0:1788572583.547615:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360717952 00000100:00100000:0.0:1788572583.547781:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5061 00000100:00100000:0.0:1788572583.547795:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37420:x1875453360717952:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.547881:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572583.547918:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb1668f8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b56 expref: 60 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.547943:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572583.547952:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572583.548361:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.548383:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.548397:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572583.548642:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c14bd788 reply state 00000000d05cd077 transno 4294971147 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.548654:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572583.548881:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37420:x1875453360717952:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1087us (1410us total) trans 4294971147 rc 0/0 00000100:00100000:0.0:1788572583.548890:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5061 00010000:00010000:1.0:1788572583.552287:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.552375:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.552385:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.552391:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572583.552397:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.552405:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.552432:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572583.552435:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572583.552440:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.552447:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb16691b lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572583.553056:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.553409:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360718080 00000100:00100000:0.0:1788572583.553423:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360718080 00000100:00100000:0.0:1788572583.553504:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5062 00000100:00100000:0.0:1788572583.553517:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37420:x1875453360718080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.553551:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572583.553576:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b6b expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.553655:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.553669:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.553702:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572583.553705:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572583.553709:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1600, rc 0) 00000100:00100000:0.0:1788572583.553791:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37420:x1875453360718080:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 276us (744us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.553799:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5062 00000100:00100000:2.0:1788572583.560055:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.560238:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360718208 00000100:00100000:1.0:1788572583.560251:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360718208 00000100:00100000:1.0:1788572583.560333:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1489 00000100:00100000:1.0:1788572583.560345:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb33f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360718208:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572583.560617:0:13272:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x62:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1788572583.560621:0:13272:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x62:0x0] updating lvb mtime from disk: 0 -> 1788572577 00002000:00010000:1.0:1788572583.560624:0:13272:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x62:0x0] updating lvb atime from disk: 0 -> 1788572577 00002000:00010000:1.0:1788572583.560626:0:13272:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x62:0x0] updating lvb ctime from disk: 0 -> 1788572577 00002000:00010000:1.0:1788572583.560628:0:13272:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x62:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:1.0:1788572583.560866:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb33f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360718208:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 522us (824us total) trans 4294967685 rc 0/0 00000100:00100000:1.0:1788572583.560875:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1489 00000100:00100000:2.0:1788572583.564748:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.564949:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360718464 00000100:00100000:1.0:1788572583.564963:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360718464 00000100:00100000:3.0:1788572583.565215:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5063 00000100:00100000:3.0:1788572583.565231:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37420:x1875453360718464:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1788572583.565348:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.565371:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.565386:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.565436:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.565451:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 61 pid: 5838 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572583.565467:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382417792:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572583.565586:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5102e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572583.570499:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.570667:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360718592 00000100:00100000:1.0:1788572583.570689:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360718592 00010000:00010000:1.0:1788572583.570747:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 62 pid: 5838 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572583.570799:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1210 00000100:00100000:1.0:1788572583.570811:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360718592:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572583.570831:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572583.570839:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 62 pid: 5838 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.570866:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 62 pid: 5838 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.570898:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572583.570904:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.570917:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.570927:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.570951:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572583.571603:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360718592:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 787us (1117us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572583.571614:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1210 00010000:00010000:0.0:1788572583.571919:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646800/0x85ace67feb1668c7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb7:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b41 expref: 61 pid: 5838 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.571941:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0f800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382417792:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572583.572265:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572278:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572284:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572337:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572345:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572352:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572358:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572583.572364:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572372:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572387:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:0.0:1788572583.572391:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1788572583.572396:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.572402:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166929 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.572484:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb649180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37420:x1875453360718464:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 7258us (7752us total) trans 0 rc -22/-22 00000100:00100000:0.0:1788572583.572494:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5063 00000100:00100000:2.0:1788572583.576248:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.576448:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360718720 00000100:00100000:1.0:1788572583.576462:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360718720 00000100:00100000:1.0:1788572583.576539:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1490 00000100:00100000:1.0:1788572583.576550:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360718720:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1788572583.576611:0:23814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.576643:0:23814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.576653:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572583.576658:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1788572583.576664:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.576720:0:23814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 11 pid: 13272 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.576738:0:23814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 11 pid: 13272 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572583.576755:0:23814:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:23814:1875453382417920:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572583.576861:0:23814:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2faae40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572583.582000:0:23814:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:23814:1875453382417920:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572583.582212:0:23814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572583.584275:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572583.584698:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360718848 00000100:00100000:3.0:1788572583.584714:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360718848 00010000:00010000:3.0:1788572583.584772:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572583.584817:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1211 00000100:00100000:3.0:1788572583.584830:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360718848:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572583.584861:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572583.584891:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.584907:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.584941:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a0bc467/0x85ace67feb166906 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b64 expref: 12 pid: 13272 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.585251:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572583.585257:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572583.585266:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.585280:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.585431:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572583.585517:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360718848:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 689us (1257us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.585526:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1211 00010000:00010000:2.0:1788572583.586187:0:23814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586213:0:23814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586259:0:23814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586268:0:23814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586276:0:23814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586283:0:23814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647600 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572583.586288:0:23814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586457:0:23814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586482:0:23814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.586491:0:23814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075bcdcc2/0x85ace67feb166930 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572583.586586:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb648e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360718720:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 10035us (10350us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572583.586602:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1490 00000100:00100000:3.0:1788572583.593141:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.593362:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360719104 00000100:00100000:0.0:1788572583.593375:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360719104 00000100:00100000:0.0:1788572583.593458:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5064 00000100:00100000:0.0:1788572583.593471:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37420:x1875453360719104:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1788572583.593594:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.593625:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.593644:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.593996:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.594058:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.594261:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.594310:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 61 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.594515:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 61 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572583.594535:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382418048:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572583.594660:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b0b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572583.600980:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.601812:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360719232 00000100:00100000:2.0:1788572583.601828:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360719232 00010000:00010000:2.0:1788572583.601895:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 62 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572583.601952:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1212 00000100:00100000:2.0:1788572583.601966:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0faf050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360719232:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572583.602031:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1788572583.602034:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382418048:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572583.602040:0:11655:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 62 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.602053:0:11655:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 62 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.602081:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb166922 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b6b expref: 62 pid: 5839 timeout: 1031 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.603553:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.605404:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:2.0:1788572583.605421:0:11655:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.605439:0:11655:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.605449:0:11655:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.605794:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572583.605902:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0faf050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360719232:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 3931us (4937us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.605915:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1212 00010000:00010000:1.0:1788572583.608404:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.608426:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788572583.608701:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971148 00000001:00080000:1.0:1788572583.608889:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971149 00000001:00080000:1.0:1788572583.608948:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971150 00000100:00100000:1.0:1788572583.609301:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382418176:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572583.609503:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572583.609556:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c5f5ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572583.609798:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382418176 00000100:00100000:3.0:1788572583.609808:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382418176 00000100:00100000:3.0:1788572583.609877:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:3.0:1788572583.609889:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382418176:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572583.610458:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382418176:12345-0@lo:1000:mdt00_000.0 Request processed in 568us (953us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.610468:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1788572583.610668:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967683 gen 2 00000004:00080000:1.0:1788572583.610688:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c4920e00, transno 4294967677 00000004:00080000:1.0:1788572583.610703:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9626c4922680, transno 4294967678 00000004:00080000:1.0:1788572583.610706:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff962801f91880, transno 4294967679 00000004:00080000:1.0:1788572583.610711:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967681 committed transno 4294967683 00000004:00080000:1.0:1788572583.610734:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f56f1500, transno 4294967683 00000100:00100000:1.0:1788572583.610743:0:5837:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9627f56f0000 x1875453382417408/t4294967684(4294967684) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1788572599 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572583.610758:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382418176:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1788572583.613246:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971151 00000100:00100000:1.0:1788572583.613443:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c10a1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382418304:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1788572583.613537:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572583.613594:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca885600 going to sleep for 11 seconds 00000100:00100000:2.0:1788572583.613904:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382418304 00000100:00100000:2.0:1788572583.613929:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382418304 00000100:00100000:2.0:1788572583.614039:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:2.0:1788572583.614053:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382418304:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572583.614522:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5837:x1875453382418304:12345-0@lo:1000:mdt00_000.0 Request processed in 472us (984us total) trans 4294967686 rc 0/0 00000100:00100000:2.0:1788572583.614533:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:1.0:1788572583.615493:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c10a1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382418304:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1788572583.615705:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971152 00010000:00010000:1.0:1788572583.615765:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615778:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615787:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615794:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572583.615799:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615809:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615828:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572583.615832:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572583.615837:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615845:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb16693e lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb7:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615888:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615894:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615900:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615905:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572583.615908:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615914:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615922:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:1.0:1788572583.615925:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788572583.615933:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.615946:0:5837:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.616218:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.616298:0:5837:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0600/0x85ace67feb16686c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b2c expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572583.616314:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382418432:192.168.202.40@tcp:105: 00000100:00100000:2.0:1788572583.617624:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca885600 going to sleep for 11 seconds 00000004:00080000:3.0:1788572583.620844:0:6872:0:(osp_sync.c:1250:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 1 in progress, 0 in flight, 4 done 00000100:00100000:2.0:1788572583.624644:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f92300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382418432:192.168.202.40@tcp:105: 00010000:00010000:2.0:1788572583.624705:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624722:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb166865 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624768:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624786:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624801:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624813:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624820:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624833:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624839:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffe0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572583.624844:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624852:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624866:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:2.0:1788572583.624869:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572583.624873:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624879:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0000/0x85ace67feb166945 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624890:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624896:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624902:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624906:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572583.624909:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624918:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624925:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb2:0x0].0x0 (ffff9627ff5abb00) 00010000:00010000:2.0:1788572583.624928:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1788572583.624930:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.624936:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166937 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb2:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572583.625753:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572583.625760:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:1.0:1788572583.626079:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000040:00080000:3.0:1788572583.626297:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 9 00000040:00080000:3.0:1788572583.626308:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1788572583.626551:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37420:x1875453360719104:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 32109us (32455us total) trans 4294971153 rc 0/0 00000100:00100000:2.0:1788572583.626570:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5064 00000020:00080000:2.0:1788572583.626579:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:2.0:1788572583.626617:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360719360 00000100:00100000:2.0:1788572583.626627:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360719360 00000100:00100000:2.0:1788572583.626693:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1106 00000100:00100000:2.0:1788572583.626703:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37416:x1875453360719360:12345-192.168.202.40@tcp:49:md5sum.0 00000100:00100000:2.0:1788572583.627554:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37416:x1875453360719360:12345-192.168.202.40@tcp:49:md5sum.0 Request processed in 845us (1469us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572583.627565:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1106 00000100:00100000:1.0:1788572583.636030:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.636118:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360719616 00000100:00100000:2.0:1788572583.636137:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360719616 00000100:00100000:2.0:1788572583.636226:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1107 00000100:00100000:2.0:1788572583.636240:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37420:x1875453360719616:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1788572583.637526:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xb7:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000100:00100000:1.0:1788572583.639264:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000001:00080000:2.0:1788572583.639397:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971154 00010000:00010000:2.0:1788572583.639501:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.639524:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.639539:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.639549:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.639556:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.639563:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.639588:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788572583.639634:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572583.639643:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.639650:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffe0200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572583.639665:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.639675:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.639696:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb7:0x0].0x0 (ffff9627e3306700) 00010000:00010000:0.0:1788572583.639699:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788572583.639704:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 1/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.639711:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffe0200/0x85ace67feb16694c lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572583.639734:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572583.639736:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572583.639737:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37420:x1875453360719616:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 3489us (3740us total) trans 4294971155 rc 0/0 00010000:00010000:0.0:1788572583.639738:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572583.639750:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1107 00000020:00080000:2.0:1788572583.639757:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788572583.640095:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360719744 00000100:00100000:3.0:1788572583.640107:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360719744 00000100:00100000:3.0:1788572583.640161:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1491 00000100:00100000:3.0:1788572583.640172:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+72:37416:x1875453360719744:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572583.640190:0:6679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572583.640221:0:6679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064d42a5b/0x85ace67feb166953 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b72 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572583.640253:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x62:0x0] initial LVB size: 1048576, mtime: 0x6a9b73a1, atime: 0x6a9b73a1, ctime: 0x6a9b73a1, blocks: 0x803 00010000:00010000:3.0:1788572583.640275:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=931s v=(0 0) 00010000:00010000:3.0:1788572583.640281:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572583.640303:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064d42a5b/0x85ace67feb166953 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b72 expref: 73 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.640321:0:6679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000064d42a5b/0x85ace67feb166953 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b72 expref: 73 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.640336:0:6679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4600, rc 0) 00000100:00100000:3.0:1788572583.640386:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:37416:x1875453360719744:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 217us (1140us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.640393:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1491 00000100:00100000:1.0:1788572583.644563:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.644792:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360719872 00000100:00100000:2.0:1788572583.644808:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360719872 00002000:00100000:2.0:1788572583.644880:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x62:0x0] (0->32767) req@ffff9627f56f1500 x1875453360719872/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:244/0 lens 488/0 e 0 to 0 dl 1788572594 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572583.644893:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f56f1500 with x1875453360719872 ext(0->32767) 00002000:00010000:2.0:1788572583.644899:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f56f1500 00000100:00100000:2.0:1788572583.644942:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 389 00000100:00100000:2.0:1788572583.644956:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360719872:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:3.0:1788572583.655089:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360719872:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 10118us (10529us total) trans 0 rc 32768/32768 00000100:00100000:3.0:1788572583.655108:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 389 00002000:00100000:3.0:1788572583.655126:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x62:0x0] (0->32767) req@ffff9627f56f1500 x1875453360719872/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:244/0 lens 488/440 e 0 to 0 dl 1788572594 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572583.655138:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f56f1500 with x1875453360719872 ext(0->32767) 00002000:00010000:3.0:1788572583.655146:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627f56f1500 00000100:00100000:0.0:1788572583.663885:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.663990:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360720000 00000100:00100000:1.0:1788572583.664038:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360720000 00002000:00100000:1.0:1788572583.664119:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x62:0x0] (32768->1048575) req@ffff9627dfdbce00 x1875453360720000/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:244/0 lens 488/0 e 0 to 0 dl 1788572594 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572583.664139:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbce00 with x1875453360720000 ext(32768->1048575) 00002000:00010000:1.0:1788572583.664146:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627dfdbce00 00000100:00100000:1.0:1788572583.664189:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 390 00000100:00100000:1.0:1788572583.664210:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360720000:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:1.0:1788572583.692689:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2349:x1875453360720000:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 28466us (28796us total) trans 0 rc 1015808/1015808 00000100:00100000:1.0:1788572583.692708:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 390 00002000:00100000:1.0:1788572583.693354:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x62:0x0] (32768->1048575) req@ffff9627dfdbce00 x1875453360720000/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:244/0 lens 488/440 e 0 to 0 dl 1788572594 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788572583.693371:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627dfdbce00 with x1875453360720000 ext(32768->1048575) 00002000:00010000:1.0:1788572583.693379:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9627dfdbce00 00000100:00100000:0.0:1788572583.708576:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572583.708812:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360720256 00000100:00100000:2.0:1788572583.708827:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360720256 00000100:00100000:2.0:1788572583.708908:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1108 00000100:00100000:2.0:1788572583.708920:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37416:x1875453360720256:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:2.0:1788572583.710105:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37416:x1875453360720256:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 1176us (1534us total) trans 4294971156 rc 0/0 00000100:00100000:2.0:1788572583.710117:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1108 00000100:00100000:3.0:1788572583.742586:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3305:1875453382418560:0@lo:400:kworker.0 00000100:00100000:0.0:1788572583.742619:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3303:1875453382418816:0@lo:400:kworker.0 00000100:00100000:2.0:1788572583.742698:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382418944:0@lo:400:kworker.0 00000100:00100000:1.0:1788572583.743084:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382418688:0@lo:400:kworker.0 00000100:00100000:0.0:1788572583.746248:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.746878:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.746936:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382419072:0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.747027:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.747073:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382418560 00000100:00100000:3.0:1788572583.747084:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382418560 00000100:00100000:3.0:1788572583.747157:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:3.0:1788572583.747168:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0f1c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3305:x1875453382418560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.747370:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0f1c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3305:x1875453382418560:12345-0@lo:400:kworker.0 Request processed in 203us (496us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.747380:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:3.0:1788572583.747461:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382419072 00000100:00100000:3.0:1788572583.747466:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382419072 00000100:00100000:3.0:1788572583.747526:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:3.0:1788572583.747533:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382419072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.747571:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382419072:12345-0@lo:400:kworker.0 Request processed in 39us (544us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.747577:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00080000:3.0:1788572583.747595:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788572583.747639:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382419072:0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.747649:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3305:1875453382418560:0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.747699:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382418816 00000100:00100000:3.0:1788572583.747703:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382418816 00000100:00100000:3.0:1788572583.747739:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:3.0:1788572583.747746:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1875453382418816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.747787:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1875453382418816:12345-0@lo:400:kworker.0 Request processed in 42us (2311us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.747794:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00080000:3.0:1788572583.747812:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572583.748350:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3303:1875453382418816:0@lo:400:kworker.0 00000100:00100000:2.0:1788572583.748458:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.749733:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382418944 00000100:00100000:3.0:1788572583.749746:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382418944 00000100:00100000:3.0:1788572583.749844:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:3.0:1788572583.749856:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382418944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.749943:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382418944:12345-0@lo:400:kworker.0 Request processed in 90us (1485us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.749951:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00080000:2.0:1788572583.750067:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572583.750107:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382418944:0@lo:400:kworker.0 00000100:00100000:0.0:1788572583.755850:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572583.757052:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382418688 00000100:00100000:3.0:1788572583.757080:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382418688 00000100:00100000:3.0:1788572583.757169:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:3.0:1788572583.757181:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382418688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.757309:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382418688:12345-0@lo:400:kworker.0 Request processed in 131us (1472us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.757373:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfdbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382418688:0@lo:400:kworker.0 00000100:00100000:3.0:1788572583.757572:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00100000:2.0:1788572583.758935:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572583.759331:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360720512 00000100:00100000:3.0:1788572583.759343:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360720512 00000100:00100000:3.0:1788572583.759420:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5065 00000100:00100000:3.0:1788572583.759432:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37422:x1875453360720512:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:3.0:1788572583.759463:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572583.759509:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16695a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b79 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.759616:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.759635:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.759648:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.759666:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.759673:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.759679:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760127:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb16696f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760144:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb16696f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760152:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb16696f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572583.760163:0:23058:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00010000:00010000:3.0:1788572583.760303:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760311:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760318:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760324:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572583.760329:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760338:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760352:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572583.760356:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572583.760361:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760368:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166961 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760388:0:23058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760394:0:23058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760400:0:23058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760405:0:23058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572583.760408:0:23058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760414:0:23058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760421:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x95494168 (ffff9626c4b10100) 00010000:00010000:3.0:1788572583.760423:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572583.760426:0:23058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760431:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb166968 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0x95494168 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760489:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16695a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b79 expref: 250 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760504:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb16696f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b79 expref: 249 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.760518:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627c5160000) 00010000:00010000:3.0:1788572583.760520:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788572583.760523:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4a00, rc 0) 00000100:00100000:3.0:1788572583.760611:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37422:x1875453360720512:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 1181us (1688us total) trans 4294971157 rc 0/0 00000100:00100000:3.0:1788572583.760619:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5065 00000100:00100000:0.0:1788572583.771916:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572583.772729:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360720768 00000100:00100000:1.0:1788572583.772745:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360720768 00000100:00100000:1.0:1788572583.772851:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 383 00000100:00100000:1.0:1788572583.772864:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37422:x1875453360720768:12345-192.168.202.40@tcp:103:md5sum.0 00010000:00010000:1.0:1788572583.772885:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788572583.772930:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4a00/0x85ace67feb16696f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b79 expref: 249 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.772959:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb16682d lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1af4 expref: 248 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572583.772980:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9627c5160000) 00010000:00010000:1.0:1788572583.772995:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572583.779623:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37422:x1875453360720768:12345-192.168.202.40@tcp:103:md5sum.0 Request processed in 6735us (7696us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572583.779646:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 383 00000100:00100000:0.0:1788572583.788508:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572583.788661:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360720896 00000100:00100000:3.0:1788572583.788792:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360720896 00000100:00100000:3.0:1788572583.789237:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5066 00000100:00100000:3.0:1788572583.789252:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37422:x1875453360720896:12345-192.168.202.40@tcp:58:md5sum.0 00010000:00010000:3.0:1788572583.789467:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.789492:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572583.789508:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572583.790590:0:22536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].177, 136 off32056 00000100:00100000:3.0:1788572583.792387:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37422:x1875453360720896:12345-192.168.202.40@tcp:58:md5sum.0 Request processed in 3121us (3877us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572583.792406:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5066 00000100:00100000:2.0:1788572583.794836:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572583.795237:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360721024 00000100:00100000:0.0:1788572583.795254:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360721024 00000100:00100000:2.0:1788572583.795514:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5067 00000100:00100000:2.0:1788572583.795528:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37422:x1875453360721024:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572583.795556:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572583.795585:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b80 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.795663:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 2/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.795692:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 4/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.795720:0:5837:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.795743:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.795760:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:2.0:1788572583.795763:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572583.795767:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572583.795776:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9000, rc 0) 00000100:00100000:2.0:1788572583.795854:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37422:x1875453360721024:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 329us (1031us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572583.795862:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5067 00010000:00010000:2.0:1788572584.065283:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1788572584.250480:0:7938:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 932319471475ns, next at 933319467685ns, rc = 0 00000100:00100000:2.0:1788572584.250636:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572584.250748:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382419200:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788572584.250879:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572584.252223:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382419200 00000100:00100000:1.0:1788572584.252248:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382419200 00000100:00100000:1.0:1788572584.252945:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1788572584.254201:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1875453382419200:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788572584.254277:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 200605696 pending 0 free 3854565376 avail 3852468224 00000100:00100000:1.0:1788572584.255719:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1875453382419200:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1544us (4830us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572584.255735:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:2.0:1788572584.256996:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382419200:0@lo:13:osp-pre-1-0.0 00000040:00080000:3.0:1788572584.636544:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1788572584.636559:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1788572584.636630:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 178 count 10 00000040:00080000:3.0:1788572584.636639:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1788572584.636836:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627ffc0ed80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382419328:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1788572584.636976:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ff548780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572584.646183:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627ffc0ed80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382419328:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572584.646761:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.647220:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360721152 00000100:00100000:1.0:1788572584.647231:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360721152 00000100:00100000:1.0:1788572584.647338:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5068 00000100:00100000:1.0:1788572584.647351:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37424:x1875453360721152:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572584.647400:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb3:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572584.647762:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37424:x1875453360721152:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 413us (1003us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572584.649189:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5068 00000100:00080000:2.0:1788572584.650730:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572584.664832:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572584.665061:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360721280 00000100:00100000:3.0:1788572584.665081:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360721280 00000100:00100000:3.0:1788572584.665180:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5069 00000100:00100000:3.0:1788572584.665194:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37424:x1875453360721280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572584.665329:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665357:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665373:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665392:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665400:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665406:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665437:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665447:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665533:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572584.665594:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00000004:00200000:3.0:1788572584.665600:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572584.665621:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665637:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665645:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665653:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665660:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572584.665667:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665677:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665720:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572584.665724:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572584.665730:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665738:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4200/0x85ace67feb16698b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665758:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665764:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.665769:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.666402:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572584.666405:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.666413:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.666426:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x1d5837b6 (ffff9626c4b10700) 00010000:00010000:3.0:1788572584.666429:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572584.666433:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.666439:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5000/0x85ace67feb166992 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x1d5837b6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667617:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667649:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667657:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667666:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572584.667671:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667680:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667699:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:3.0:1788572584.667701:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572584.667705:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.667724:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4e00/0x85ace67feb166999 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572584.671339:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37424:x1875453360721280:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 6129us (6512us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572584.671359:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5069 00000100:00100000:0.0:1788572584.685394:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572584.685513:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360721536 00000100:00100000:3.0:1788572584.685527:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360721536 00000100:00100000:3.0:1788572584.685604:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5070 00000100:00100000:3.0:1788572584.685615:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37424:x1875453360721536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572584.685651:0:5839:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572584.685681:0:5839:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669a0 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b87 expref: -99 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.685833:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.685858:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.685872:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572584.685885:0:5839:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb3:0x0]: returning size 0 00010000:00010000:3.0:1788572584.686064:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686073:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686080:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686086:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572584.686091:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686100:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686114:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:3.0:1788572584.686117:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788572584.686122:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686128:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669a7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb3:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686197:0:5839:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669a0 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b87 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686209:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572584.686212:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572584.686216:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669a0 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b87 expref: 69 pid: 5839 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.686227:0:5839:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9626c24e4400, rc 0) 00000100:00100000:3.0:1788572584.686305:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37424:x1875453360721536:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 691us (925us total) trans 4294971158 rc 301/301 00000100:00100000:3.0:1788572584.686314:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5070 00000100:00100000:2.0:1788572584.692622:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.694319:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360721664 00000100:00100000:1.0:1788572584.694342:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360721664 00000100:00100000:1.0:1788572584.694444:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5071 00000100:00100000:1.0:1788572584.694465:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37424:x1875453360721664:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1788572584.694525:0:5838:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb3:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1788572584.694700:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37424:x1875453360721664:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 238us (2093us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572584.694709:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5071 00000100:00100000:1.0:1788572584.703151:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.703476:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360721920 00000100:00100000:0.0:1788572584.703491:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360721920 00000100:00100000:0.0:1788572584.703568:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5072 00000100:00100000:0.0:1788572584.703579:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37424:x1875453360721920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572584.703605:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572584.703640:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1669b5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b8e expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.714391:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1669bc lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.714426:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1669bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.714452:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1669bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788572584.714472:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb8:0x0]: returning size 0 00010000:00010000:1.0:1788572584.714591:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1400/0x85ace67feb1669b5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b8e expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.714621:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1669bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b8e expref: 70 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.714642:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:1.0:1788572584.714646:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788572584.714650:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647200, rc 0) 00000100:00100000:1.0:1788572584.714766:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37424:x1875453360721920:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 11187us (11628us total) trans 4294971159 rc 0/0 00000100:00100000:1.0:1788572584.714784:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5072 00000100:00100000:2.0:1788572584.721530:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572584.721719:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360722176 00000100:00100000:3.0:1788572584.721735:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360722176 00000100:00100000:3.0:1788572584.721814:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5073 00000100:00100000:3.0:1788572584.721827:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37424:x1875453360722176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572584.721904:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572584.721950:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647200/0x85ace67feb1669bc lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b8e expref: 70 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.721973:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627ff5ab100) 00010000:00010000:3.0:1788572584.721985:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572584.722086:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.722103:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.722116:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.722133:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.722140:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.722148:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725697:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725847:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725856:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725864:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572584.725869:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725879:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725901:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572584.725904:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572584.725910:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725918:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4400/0x85ace67feb1669ca lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725935:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725941:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725947:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725952:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572584.725954:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725960:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725968:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xc9ae164f (ffff9626c4b10700) 00010000:00010000:3.0:1788572584.725971:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572584.725974:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.725980:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669d1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xc9ae164f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572584.728112:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37424:x1875453360722176:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 6271us (6581us total) trans 0 rc -2/-2 00000100:00100000:3.0:1788572584.728128:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5073 00000100:00100000:1.0:1788572584.734644:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572584.734743:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360722304 00000100:00100000:3.0:1788572584.734757:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360722304 00000100:00100000:0.0:1788572584.735280:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5074 00000100:00100000:0.0:1788572584.735299:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37424:x1875453360722304:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572584.735331:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572584.735364:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669d8 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b95 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.735416:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.735431:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.735456:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.735588:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669d8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 71 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.735611:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 70 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.735620:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:0.0:1788572584.735623:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572584.735628:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1200, rc 0) 00000100:00100000:0.0:1788572584.735731:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37424:x1875453360722304:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 438us (1109us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.735739:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5074 00000100:00100000:1.0:1788572584.742760:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572584.744071:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360722560 00000100:00100000:3.0:1788572584.744089:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360722560 00000100:00100000:3.0:1788572584.744178:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5075 00000100:00100000:3.0:1788572584.744192:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37424:x1875453360722560:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572584.744315:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.744340:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572584.744356:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572584.744653:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f570ed86 reply state 000000004a70228e transno 4294971160 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572584.744664:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572584.745426:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37424:x1875453360722560:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1232us (2675us total) trans 4294971160 rc 0/0 00000100:00100000:3.0:1788572584.745438:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5075 00000100:00100000:2.0:1788572584.753059:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.753154:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360722688 00000100:00100000:0.0:1788572584.753182:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360722688 00000100:00100000:0.0:1788572584.753259:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5076 00000100:00100000:0.0:1788572584.753272:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37424:x1875453360722688:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572584.753305:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572584.753331:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1b9c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.753417:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 73 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.753437:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 73 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.753459:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:0.0:1788572584.753463:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572584.753468:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1600, rc 0) 00000100:00100000:0.0:1788572584.753553:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360722688:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 283us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.753560:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5076 00010000:00010000:1.0:1788572584.760191:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.760215:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.760225:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.760525:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572584.760532:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.760541:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.760567:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:1.0:1788572584.760570:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1788572584.760577:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.760585:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4800/0x85ace67feb1669e6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572584.762035:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.762592:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360722816 00000100:00100000:1.0:1788572584.762607:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360722816 00000100:00100000:1.0:1788572584.762698:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1109 00000100:00100000:1.0:1788572584.762719:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13dc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37424:x1875453360722816:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1788572584.763951:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13dc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37424:x1875453360722816:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 Request processed in 1227us (2153us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572584.763965:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1109 00000100:00100000:2.0:1788572584.783737:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.783921:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360723072 00000100:00100000:1.0:1788572584.783936:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360723072 00000100:00100000:2.0:1788572584.784429:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1110 00000100:00100000:2.0:1788572584.784460:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37424:x1875453360723072:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572584.785188:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360723072:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 743us (1461us total) trans 4294971161 rc 0/0 00000100:00100000:2.0:1788572584.785199:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1110 00000100:00100000:3.0:1788572584.805276:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.805391:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360723328 00000100:00100000:0.0:1788572584.805404:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360723328 00000100:00100000:0.0:1788572584.805491:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5077 00000100:00100000:0.0:1788572584.805504:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360723328:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572584.805670:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360723328:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 168us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.805681:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5077 00000100:00100000:2.0:1788572584.810100:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.810309:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360723584 00000100:00100000:1.0:1788572584.810324:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360723584 00000100:00100000:1.0:1788572584.810527:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5078 00000100:00100000:1.0:1788572584.810543:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360723584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1788572584.810576:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788572584.810610:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1669f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1ba3 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.811827:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.811859:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.811870:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.811930:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.811949:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 74 pid: 5837 timeout: 1032 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572584.811966:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627f56f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382419456:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572584.812105:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572584.821352:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627f56f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382419456:192.168.202.40@tcp:104: 00010000:00010000:1.0:1788572584.821570:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572584.823122:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.823390:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360723712 00000100:00100000:1.0:1788572584.823403:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360723712 00010000:00010000:1.0:1788572584.823485:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 75 pid: 5837 timeout: 1032 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572584.823539:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1213 00000100:00100000:1.0:1788572584.823551:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360723712:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572584.823572:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572584.823580:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 75 pid: 5837 timeout: 1032 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.823593:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 75 pid: 5837 timeout: 1032 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.823620:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1600/0x85ace67feb1669ed lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b9c expref: 75 pid: 5837 timeout: 1032 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.823641:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:1.0:1788572584.823647:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.823659:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.823669:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572584.823715:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572584.823787:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360723712:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 237us (679us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572584.823795:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1213 00010000:00010000:0.0:1788572584.823983:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.823994:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572584.824064:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb8:0x0]: returning size 0 00000004:00080000:0.0:1788572584.824544:0:22536:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x62:0x0], index 1: 0 00010000:00010000:0.0:1788572584.824773:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824783:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824791:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824799:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb646400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572584.824806:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824817:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824836:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:0.0:1788572584.824840:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572584.824846:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824853:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb646400/0x85ace67feb1669fb lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824923:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1669f4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ba3 expref: 74 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824934:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572584.824937:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572584.824941:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647400/0x85ace67feb1669f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1ba3 expref: 74 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.824950:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627fb647400, rc 0) 00000100:00100000:0.0:1788572584.825036:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360723584:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 14466us (14919us total) trans 4294971162 rc 301/301 00000100:00100000:0.0:1788572584.825045:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5078 00000100:00100000:2.0:1788572584.836734:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572584.837105:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360723968 00000100:00100000:1.0:1788572584.837120:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360723968 00000100:00100000:1.0:1788572584.837204:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1111 00000100:00100000:1.0:1788572584.837216:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37424:x1875453360723968:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572584.842823:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37424:x1875453360723968:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 5590us (6095us total) trans 4294971163 rc 0/0 00000100:00100000:1.0:1788572584.842844:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1111 00000100:00100000:2.0:1788572584.855556:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.855856:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724096 00000100:00100000:0.0:1788572584.855871:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724096 00000100:00100000:0.0:1788572584.855954:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5079 00000100:00100000:0.0:1788572584.855967:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37424:x1875453360724096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572584.856001:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572584.856062:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166a09 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1baa expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.856173:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166a10 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.856191:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166a10 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.856205:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166a10 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572584.856252:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb8:0x0]: returning size 0 00000004:00200000:0.0:1788572584.856256:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788572584.856282:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166a10 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.856318:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1c00/0x85ace67feb166a09 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1baa expref: 76 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.856342:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166a10 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1baa expref: 75 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.856353:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:0.0:1788572584.856356:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1788572584.856361:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0800, rc 0) 00000100:00100000:0.0:1788572584.856508:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb473b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37424:x1875453360724096:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 543us (966us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.856517:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5079 00000100:00100000:2.0:1788572584.859722:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.859907:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724224 00000100:00100000:0.0:1788572584.859921:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724224 00000100:00100000:0.0:1788572584.860041:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1492 00000100:00100000:0.0:1788572584.860057:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360724224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572584.860081:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572584.860107:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bb1 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572584.860142:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x62:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1788572584.860179:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=932s v=(0 0) 00010000:00010000:0.0:1788572584.860186:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572584.860200:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.860232:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.860251:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0000, rc 0) 00000100:00100000:0.0:1788572584.860396:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360724224:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 342us (696us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.860404:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1492 00000100:00080000:2.0:1788572584.860872:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788572584.867924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.875188:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724480 00000100:00100000:0.0:1788572584.875213:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724480 00000100:00100000:0.0:1788572584.875319:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5080 00000100:00100000:0.0:1788572584.875332:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37424:x1875453360724480:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572584.876078:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37424:x1875453360724480:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 742us (8162us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.876090:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5080 00000100:00100000:3.0:1788572584.882338:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572584.883198:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724608 00000100:00100000:2.0:1788572584.883217:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724608 00000100:00100000:2.0:1788572584.883304:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1493 00000100:00100000:2.0:1788572584.883318:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:37424:x1875453360724608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572584.883348:0:23814:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572584.883375:0:23814:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bb8 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572584.883429:0:23814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572584.883449:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572584.883454:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572584.883461:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 12 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572584.883520:0:23814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572584.883537:0:23814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 1032 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572584.883556:0:23814:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:23814:1875453382419584:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572584.883695:0:23814:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec075840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572584.890965:0:23814:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb472680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:23814:1875453382419584:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572584.891050:0:23814:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 3/0,0 mode: --/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 12 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.891077:0:23814:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8200, rc 0) 00000100:00100000:3.0:1788572584.893618:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:37424:x1875453360724608:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 10279us (11273us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572584.893831:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1493 00000100:00100000:1.0:1788572584.900988:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572584.901334:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724736 00000100:00100000:3.0:1788572584.901348:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724736 00010000:00010000:3.0:1788572584.901620:0:11655:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 1032 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572584.910193:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1214 00000100:00100000:0.0:1788572584.910231:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360724736:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572584.910265:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572584.910304:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 1032 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.910327:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 1032 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.910359:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ac59cad4/0x85ace67feb166a17 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb1 expref: 12 pid: 6682 timeout: 1032 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.910382:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572584.910387:0:5824:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572584.910397:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.910409:0:5824:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572584.910478:0:5824:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572584.910494:0:5824:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382419712:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572584.910622:0:5824:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d12b0b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572584.919490:0:5824:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5824:1875453382419712:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572584.919528:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572584.919604:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360724736:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 9394us (18904us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572584.919620:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1214 00000100:00100000:1.0:1788572584.950279:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572584.950663:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724864 00000100:00100000:0.0:1788572584.950676:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724864 00002000:00100000:0.0:1788572584.950757:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x62:0x0] (0->1048575) req@ffff9627f56f1f80 x1875453360724864/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:245/0 lens 488/0 e 0 to 0 dl 1788572595 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572584.950779:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f56f1f80 with x1875453360724864 ext(0->1048575) 00002000:00010000:0.0:1788572584.950786:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627f56f1f80 00000100:00100000:0.0:1788572584.950829:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 391 00000100:00100000:0.0:1788572584.950840:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360724864:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1788572585.011288:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360724864:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 60431us (61011us total) trans 4294967683 rc 0/0 00000100:00100000:2.0:1788572585.011308:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 391 00002000:00100000:2.0:1788572585.011334:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x62:0x0] (0->1048575) req@ffff9627f56f1f80 x1875453360724864/t4294967683(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:245/0 lens 488/448 e 0 to 0 dl 1788572595 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572585.011347:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627f56f1f80 with x1875453360724864 ext(0->1048575) 00002000:00010000:2.0:1788572585.011356:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627f56f1f80 00000100:00100000:3.0:1788572585.019257:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572585.019562:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360724992 00000100:00100000:0.0:1788572585.019576:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360724992 00000100:00100000:0.0:1788572585.019730:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5081 00000100:00100000:0.0:1788572585.019743:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37424:x1875453360724992:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00010000:00010000:1.0:1788572585.088860:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1788572585.118894:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971162 is committed 00000001:00080000:1.0:1788572585.118940:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971163 is committed 00000004:00080000:1.0:1788572585.118947:0:5859:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 45 committed 00000020:00080000:1.0:1788572585.118967:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627c2faae40 refcount 2 committed 0 result 0 batchid 149 00000020:00080000:1.0:1788572585.118974:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8af800 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627c8257840 00000020:00080000:1.0:1788572585.118977:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627ca8b1100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9626c4a7d800 00000020:00080000:1.0:1788572585.119110:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 149 committed batchid 147 00000020:00080000:1.0:1788572585.119384:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 147 00000100:00100000:1.0:1788572585.119410:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382419840:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1788572585.119514:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788572585.119632:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382419840 00000100:00100000:1.0:1788572585.119642:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382419840 00000100:00100000:1.0:1788572585.119723:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1494 00000100:00100000:1.0:1788572585.119734:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1875453382419840:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1788572585.119766:0:6679:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:96 count 1 00002000:00010000:1.0:1788572585.119814:0:6679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x60:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1788572585.119823:0:6679:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.119848:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=933s v=(0 0) 00010000:00010000:1.0:1788572585.119853:0:6679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x60:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788572585.119862:0:6679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.119880:0:6679:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.119892:0:6679:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.119901:0:6679:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.119921:0:6679:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788572585.120513:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3303:x1875453382419840:12345-0@lo:6:osp-syn-1-0.0 Request processed in 777us (995us total) trans 4294967684 rc 0/0 00000100:00100000:1.0:1788572585.120522:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1494 00000100:00080000:1.0:1788572585.120557:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1788572585.120571:0:5826:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572585.120578:0:5826:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.120589:0:5826:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572585.120595:0:5826:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.120606:0:5826:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.120627:0:5826:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.120636:0:5826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006a0bc467/0x85ace67feb166a25 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x60:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.120659:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572585.120661:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572585.120663:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1788572585.120708:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff962801f91c00/2, rc 0, transno 4294967684 00000100:00100000:1.0:1788572585.120716:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382419840:0@lo:6:osp-syn-1-0.0 00000001:00080000:1.0:1788572585.120761:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971156 is committed 00000001:00080000:1.0:1788572585.120798:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971157 is committed 00000020:00080000:1.0:1788572585.120806:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 148 batchid updated 00000100:00100000:3.0:1788572585.122923:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37424:x1875453360724992:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 103165us (103662us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572585.122940:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5081 00000100:00100000:3.0:1788572585.126880:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.127127:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360725120 00000100:00100000:1.0:1788572585.127154:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360725120 00000100:00100000:1.0:1788572585.127246:0:13272:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1495 00000100:00100000:1.0:1788572585.127259:0:13272:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360725120:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1788572585.253457:0:13272:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ffc0f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2347:x1875453360725120:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 126184us (126577us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572585.253479:0:13272:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1495 00000001:00080000:3.0:1788572585.253541:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967681 is committed 00000001:00080000:3.0:1788572585.253557:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967684 is committed 00000001:00080000:3.0:1788572585.253571:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967683 is committed 00000100:00100000:1.0:1788572585.258996:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572585.259599:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360725376 00000100:00100000:0.0:1788572585.259664:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360725376 00000100:00100000:0.0:1788572585.259748:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5082 00000100:00100000:0.0:1788572585.259761:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37424:x1875453360725376:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572585.259840:0:5837:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572585.259877:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0800/0x85ace67feb166a10 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1baa expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.259903:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:0.0:1788572585.259909:0:5837:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572585.259950:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.260229:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.260244:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788572585.260523:0:5837:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000097d2e95d reply state 000000000b25fa1d transno 4294971164 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572585.260534:0:5837:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788572585.260685:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37424:x1875453360725376:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 925us (1718us total) trans 4294971164 rc 0/0 00000100:00100000:0.0:1788572585.260694:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5082 00000100:00100000:3.0:1788572585.263257:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572585.263451:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360725504 00000100:00100000:2.0:1788572585.263463:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360725504 00000100:00100000:2.0:1788572585.263536:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5083 00000100:00100000:2.0:1788572585.263549:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37424:x1875453360725504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572585.263581:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572585.263607:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bbf expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.263702:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 62 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.263720:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 62 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.263745:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:2.0:1788572585.263749:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572585.263753:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8800, rc 0) 00000100:00100000:2.0:1788572585.263825:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ca8c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37424:x1875453360725504:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 278us (580us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572585.263832:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5083 00010000:00010000:1.0:1788572585.264431:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.264443:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.264451:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.264457:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572585.264462:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.264471:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.264486:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:1.0:1788572585.264489:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572585.264494:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.264500:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166a2c lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572585.285677:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.286072:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360725632 00000100:00100000:1.0:1788572585.286086:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360725632 00000100:00100000:1.0:1788572585.286175:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1496 00000100:00100000:1.0:1788572585.286187:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360725632:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1788572585.286541:0:6681:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x62:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1788572585.286548:0:6681:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x62:0x0] updating lvb mtime from disk: 0 -> 1788572577 00002000:00010000:1.0:1788572585.286550:0:6681:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x62:0x0] updating lvb atime from disk: 0 -> 1788572577 00002000:00010000:1.0:1788572585.286552:0:6681:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x62:0x0] updating lvb ctime from disk: 0 -> 1788572577 00002000:00010000:1.0:1788572585.286555:0:6681:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x62:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:1.0:1788572585.286649:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360725632:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 463us (987us total) trans 4294967685 rc 0/0 00000100:00100000:1.0:1788572585.286658:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1496 00000100:00100000:2.0:1788572585.291209:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.291444:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360725888 00000100:00100000:1.0:1788572585.291458:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360725888 00000100:00100000:1.0:1788572585.291540:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5084 00000100:00100000:1.0:1788572585.291555:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37424:x1875453360725888:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1788572585.291660:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.291682:0:22536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.291694:0:22536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 4/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.291739:0:22536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.291756:0:22536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 61 pid: 23058 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572585.291771:0:22536:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c36df800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382419968:192.168.202.40@tcp:104: 00000100:00100000:1.0:1788572585.291904:0:22536:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ca8850c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572585.304627:0:22536:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c36df800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:22536:1875453382419968:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572585.304676:0:22536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572585.308267:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572585.308704:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360726016 00000100:00100000:0.0:1788572585.308724:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360726016 00010000:00010000:0.0:1788572585.308790:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 62 pid: 23058 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572585.308842:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1215 00000100:00100000:0.0:1788572585.308855:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360726016:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572585.308876:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572585.308884:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 62 pid: 23058 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.309148:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 62 pid: 23058 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.309179:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1200/0x85ace67feb1669df lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb8:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b95 expref: 62 pid: 23058 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.309200:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:0.0:1788572585.309206:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.309218:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.309226:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.309257:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788572585.309417:0:22536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309429:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309475:0:22536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309483:0:22536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309492:0:22536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309498:0:22536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627fb647c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572585.309504:0:22536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309514:0:22536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309525:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:1.0:1788572585.309529:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1788572585.309534:0:22536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.309541:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627fb647c00/0x85ace67feb166a3a lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572585.309680:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360726016:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 825us (1431us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572585.309690:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1215 00000100:00100000:1.0:1788572585.310626:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37424:x1875453360725888:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 19067us (19424us total) trans 0 rc -22/-22 00000100:00100000:1.0:1788572585.310644:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5084 00000100:00100000:2.0:1788572585.326172:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572585.333247:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360726144 00000100:00100000:3.0:1788572585.333274:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360726144 00000100:00100000:0.0:1788572585.337075:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1497 00000100:00100000:0.0:1788572585.337098:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2347:x1875453360726144:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1788572585.337175:0:23814:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.337204:0:23814:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.337221:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x62:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1788572585.337226:0:23814:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1788572585.337231:0:23814:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.337283:0:23814:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.337301:0:23814:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572585.337318:0:23814:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:23814:1875453382420096:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572585.339355:0:23814:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c2f640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572585.355179:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572585.357770:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360726272 00000100:00100000:3.0:1788572585.357802:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360726272 00010000:00010000:3.0:1788572585.357875:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 12 pid: 23814 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572585.357947:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1216 00000100:00100000:3.0:1788572585.357958:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360726272:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572585.357975:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572585.358034:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 12 pid: 23814 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.358051:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 12 pid: 23814 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.358085:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572585.358090:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572585.358098:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.358109:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.358147:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572585.358222:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2349:x1875453360726272:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 265us (3059us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572585.358231:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1216 00010000:00010000:0.0:1788572585.362740:0:23814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2d76155/0x85ace67feb166a1e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x62:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bb8 expref: 11 pid: 23814 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572585.364283:0:23814:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627c9b51c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:23814:1875453382420096:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572585.364336:0:23814:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364356:0:23814:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364373:0:23814:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364440:0:23814:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364450:0:23814:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364460:0:23814:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364467:0:23814:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1200 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:0.0:1788572585.364473:0:23814:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364482:0:23814:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364509:0:23814:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.364518:0:23814:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5127d53/0x85ace67feb166a41 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 23814 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572585.364674:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2347:x1875453360726144:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 27580us (38516us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572585.364691:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1497 00000100:00100000:2.0:1788572585.370854:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572585.371652:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360726528 00000100:00100000:3.0:1788572585.371667:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360726528 00000100:00100000:3.0:1788572585.371754:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5085 00000100:00100000:3.0:1788572585.371768:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37424:x1875453360726528:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1788572585.371901:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.371926:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.371945:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.372315:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.372330:0:5839:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.372341:0:5839:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.372872:0:5839:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 61 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.372890:0:5839:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 61 pid: 5838 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572585.372909:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382420224:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572585.373357:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572585.386927:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.387338:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360726656 00000100:00100000:1.0:1788572585.387352:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360726656 00010000:00010000:1.0:1788572585.387577:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 62 pid: 5838 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572585.387637:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1217 00000100:00100000:1.0:1788572585.387649:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360726656:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572585.387669:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572585.387676:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 62 pid: 5838 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.387693:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 62 pid: 5838 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.387722:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:1.0:1788572585.387728:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.387740:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.387750:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.387772:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572585.387849:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360726656:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 202us (938us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572585.387860:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1217 00010000:00010000:3.0:1788572585.388876:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166a33 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bbf expref: 61 pid: 5838 timeout: 1033 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572585.388913:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382420224:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572585.388945:0:5839:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.388957:0:5839:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.388964:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788572585.389314:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971165 00000001:00080000:3.0:1788572585.393239:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971166 00000001:00080000:3.0:1788572585.393345:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971167 00000100:00100000:3.0:1788572585.393764:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382420352:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572585.394076:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572585.394158:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d180 going to sleep for 11 seconds 00000100:00100000:2.0:1788572585.394685:0:5842:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382420352 00000100:00100000:2.0:1788572585.394697:0:5842:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382420352 00000100:00100000:2.0:1788572585.394999:0:5842:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1788572585.395050:0:5842:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382420352:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1788572585.395210:0:5842:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb64a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382420352:12345-0@lo:1000:mdt00_002.0 Request processed in 163us (1135us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572585.395219:0:5842:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:3.0:1788572585.395338:0:5839:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967684 gen 2 00000004:00080000:3.0:1788572585.395347:0:5839:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967682 committed transno 4294967684 00000004:00080000:3.0:1788572585.395385:0:5839:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff962801f91c00, transno 4294967684 00000100:00100000:3.0:1788572585.395425:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382420352:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1788572585.395974:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971168 00000100:00100000:3.0:1788572585.396098:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382420480:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1788572585.396170:0:5839:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788572585.398335:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572585.398575:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382420480 00000100:00100000:1.0:1788572585.398587:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382420480 00000100:00100000:0.0:1788572585.399743:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1788572585.399761:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5839:x1875453382420480:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1788572585.400236:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5839:x1875453382420480:12345-0@lo:1000:mdt00_002.0 Request processed in 478us (4064us total) trans 4294967686 rc 0/0 00000100:00100000:0.0:1788572585.400246:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:3.0:1788572585.400395:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627fb648380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5839:1875453382420480:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1788572585.400566:0:5839:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971169 00010000:00010000:3.0:1788572585.400632:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400646:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400657:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400669:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572585.400675:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400685:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400708:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9627dfeae500) 00010000:00010000:3.0:1788572585.400711:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572585.400717:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400723:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5200/0x85ace67feb166a4f lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400761:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400768:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400773:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400778:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e4000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572585.400781:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400787:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400798:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:3.0:1788572585.400800:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572585.400809:0:5839:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400822:0:5839:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400834:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.400885:0:5839:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9000/0x85ace67feb166984 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1b80 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572585.400903:0:5839:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627e7848e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382420608:192.168.202.40@tcp:105: 00000100:00100000:3.0:1788572585.401039:0:5839:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788572585.406471:0:5839:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627e7848e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5839:1875453382420608:192.168.202.40@tcp:105: 00010000:00010000:3.0:1788572585.406523:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406537:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e4000/0x85ace67feb16697d lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406583:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406600:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406626:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406640:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406647:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406655:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406662:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572585.406667:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406676:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406689:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:3.0:1788572585.406693:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572585.406697:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406704:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5800/0x85ace67feb166a56 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406714:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406721:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406728:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406734:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572585.406737:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406743:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406750:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb3:0x0].0x0 (ffff9626c4b10100) 00010000:00010000:3.0:1788572585.406753:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1788572585.406756:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.406762:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a48 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb3:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572585.407150:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37424:x1875453360726528:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 35371us (36296us total) trans 4294971170 rc 0/0 00000100:00100000:3.0:1788572585.407163:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5085 00000020:00080000:3.0:1788572585.407171:0:5839:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:2.0:1788572585.407750:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572585.407757:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572585.408289:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 10 00000040:00080000:2.0:1788572585.408301:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1788572585.416352:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572585.416791:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360726784 00000100:00100000:2.0:1788572585.416804:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360726784 00000100:00100000:2.0:1788572585.416908:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1112 00000100:00100000:2.0:1788572585.416923:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37422:x1875453360726784:12345-192.168.202.40@tcp:49:md5sum.0 00000100:00100000:2.0:1788572585.417160:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37422:x1875453360726784:12345-192.168.202.40@tcp:49:md5sum.0 Request processed in 241us (824us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572585.417168:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1112 00000100:00080000:1.0:1788572585.418480:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572585.425287:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.425858:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360727040 00000100:00100000:1.0:1788572585.425874:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360727040 00000100:00100000:1.0:1788572585.425954:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1113 00000100:00100000:1.0:1788572585.425966:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37424:x1875453360727040:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1788572585.434260:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000004:00080000:3.0:1788572585.436500:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xb8:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1788572585.436675:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971171 00010000:00010000:3.0:1788572585.436778:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.436801:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.436820:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.436830:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.436837:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.436844:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.436887:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572585.437057:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37424:x1875453360727040:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 11087us (11792us total) trans 4294971172 rc 0/0 00000100:00100000:3.0:1788572585.437070:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1113 00000020:00080000:3.0:1788572585.437076:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788572585.437118:0:8441:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572585.437125:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.437145:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9626c24e5400 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:3.0:1788572585.437150:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.437159:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.437174:0:8441:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb8:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:3.0:1788572585.437177:0:8441:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788572585.437183:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 1/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.437189:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9626c24e5400/0x85ace67feb166a5d lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.437207:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572585.437209:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572585.437211:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572585.437758:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360727168 00000100:00100000:2.0:1788572585.437771:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360727168 00000100:00100000:2.0:1788572585.437853:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1498 00000100:00100000:2.0:1788572585.437865:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+84:37422:x1875453360727168:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572585.437895:0:20911:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572585.437919:0:20911:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7d3ef84/0x85ace67feb166a64 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bc6 expref: -99 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788572585.437955:0:20911:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x62:0x0] initial LVB size: 1048576, mtime: 0x6a9b73a1, atime: 0x6a9b73a1, ctime: 0x6a9b73a1, blocks: 0x803 00010000:00010000:2.0:1788572585.437980:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=933s v=(0 0) 00010000:00010000:2.0:1788572585.437986:0:20911:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x62:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788572585.437998:0:20911:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7d3ef84/0x85ace67feb166a64 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bc6 expref: 85 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.438040:0:20911:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7d3ef84/0x85ace67feb166a64 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bc6 expref: 85 pid: 20911 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.438061:0:20911:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9c00, rc 0) 00000100:00080000:1.0:1788572585.440352:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788572585.447464:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:37422:x1875453360727168:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 9578us (13197us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572585.447482:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1498 00000100:00100000:3.0:1788572585.462349:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572585.462722:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360727296 00000100:00100000:2.0:1788572585.462737:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360727296 00002000:00100000:3.0:1788572585.463352:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x62:0x0] (0->32767) req@ffff9627e7848e00 x1875453360727296/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:246/0 lens 488/0 e 0 to 0 dl 1788572596 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572585.463531:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7848e00 with x1875453360727296 ext(0->32767) 00002000:00010000:3.0:1788572585.463539:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7848e00 00000100:00100000:3.0:1788572585.463634:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 392 00000100:00100000:3.0:1788572585.463649:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360727296:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:3.0:1788572585.470928:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360727296:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 7267us (8751us total) trans 0 rc 32768/32768 00000100:00100000:3.0:1788572585.470947:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 392 00002000:00100000:3.0:1788572585.470963:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x62:0x0] (0->32767) req@ffff9627e7848e00 x1875453360727296/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:246/0 lens 488/440 e 0 to 0 dl 1788572596 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572585.470975:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627e7848e00 with x1875453360727296 ext(0->32767) 00002000:00010000:3.0:1788572585.470984:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627e7848e00 00000100:00100000:1.0:1788572585.482960:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572585.483770:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360727424 00000100:00100000:2.0:1788572585.483788:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360727424 00002000:00100000:2.0:1788572585.483876:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x62:0x0] (32768->1048575) req@ffff9627c84eed80 x1875453360727424/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:246/0 lens 488/0 e 0 to 0 dl 1788572596 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572585.483902:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84eed80 with x1875453360727424 ext(32768->1048575) 00002000:00010000:2.0:1788572585.483910:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c84eed80 00000100:00100000:2.0:1788572585.483951:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 393 00000100:00100000:2.0:1788572585.483963:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360727424:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572585.534215:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2349:x1875453360727424:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 50236us (51255us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572585.534233:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 393 00002000:00100000:2.0:1788572585.534252:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x62:0x0] (32768->1048575) req@ffff9627c84eed80 x1875453360727424/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:246/0 lens 488/440 e 0 to 0 dl 1788572596 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572585.534264:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9627c84eed80 with x1875453360727424 ext(32768->1048575) 00002000:00010000:2.0:1788572585.534273:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9627c84eed80 00000100:00100000:3.0:1788572585.547639:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.548193:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360727680 00000100:00100000:1.0:1788572585.548210:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360727680 00000100:00100000:1.0:1788572585.548299:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1114 00000100:00100000:1.0:1788572585.548311:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37422:x1875453360727680:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:1.0:1788572585.548899:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37422:x1875453360727680:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 586us (1275us total) trans 4294971173 rc 0/0 00000100:00100000:1.0:1788572585.548911:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1114 00000100:00100000:3.0:1788572585.617233:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572585.617889:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360727936 00000100:00100000:2.0:1788572585.617903:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360727936 00000100:00100000:2.0:1788572585.617987:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5086 00000100:00100000:2.0:1788572585.618051:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37426:x1875453360727936:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:2.0:1788572585.618087:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572585.618118:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166a6b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bcd expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.618984:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.619035:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.619050:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.619071:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.619079:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.619086:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621189:0:5838:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166a80 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621218:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166a80 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621230:0:5838:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166a80 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572585.621667:0:5838:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00010000:00010000:2.0:1788572585.621835:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621845:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621854:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621860:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572585.621866:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621943:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621963:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:2.0:1788572585.621966:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572585.621974:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.621981:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166a72 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622000:0:5838:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622232:0:5838:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622253:0:5838:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622258:0:5838:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572585.622262:0:5838:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622269:0:5838:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622278:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xefa61e4e (ffff9627e3306500) 00010000:00010000:2.0:1788572585.622281:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788572585.622284:0:5838:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622290:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166a79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd0:0x0].0xefa61e4e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5838 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622378:0:5838:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166a6b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bcd expref: 250 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622395:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166a80 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bcd expref: 249 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572585.622404:0:5838:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:2.0:1788572585.622406:0:5838:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788572585.622409:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d9e00, rc 0) 00000100:00100000:2.0:1788572585.622852:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:37426:x1875453360727936:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 4851us (5631us total) trans 4294971174 rc 0/0 00000100:00100000:2.0:1788572585.622868:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5086 00000100:00100000:0.0:1788572585.651233:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.662414:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360728064 00000100:00100000:1.0:1788572585.662436:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360728064 00000100:00100000:1.0:1788572585.662536:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 384 00000100:00100000:1.0:1788572585.662549:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0fad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+249:2349:x1875453360728064:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788572585.662570:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572585.662602:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166a80 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bcd expref: 249 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.662632:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:1.0:1788572585.662639:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572585.662730:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0fad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:2349:x1875453360728064:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 183us (11513us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572585.662738:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 384 00000100:00100000:3.0:1788572585.663095:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572585.663296:0:11655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360728320 00000100:00100000:1.0:1788572585.663302:0:11655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360728320 00000100:00100000:1.0:1788572585.663358:0:11655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 385 00000100:00100000:1.0:1788572585.663366:0:11655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37426:x1875453360728320:12345-192.168.202.40@tcp:103:md5sum.0 00010000:00010000:1.0:1788572585.663377:0:11655:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788572585.663393:0:11655:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0e00/0x85ace67feb166849 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1afb expref: 248 pid: 18263 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572585.663417:0:11655:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9627d4baa100) 00010000:00010000:1.0:1788572585.663428:0:11655:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788572585.663454:0:11655:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37426:x1875453360728320:12345-192.168.202.40@tcp:103:md5sum.0 Request processed in 89us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572585.663462:0:11655:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 385 00000100:00100000:0.0:1788572585.680998:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572585.681148:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360728448 00000100:00100000:3.0:1788572585.681163:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360728448 00000100:00100000:3.0:1788572585.681240:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5087 00000100:00100000:3.0:1788572585.681254:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37426:x1875453360728448:12345-192.168.202.40@tcp:58:md5sum.0 00010000:00010000:3.0:1788572585.681412:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.681435:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572585.681449:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788572585.681872:0:9040:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].178, 136 off32192 00000100:00100000:3.0:1788572585.681993:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37426:x1875453360728448:12345-192.168.202.40@tcp:58:md5sum.0 Request processed in 740us (1024us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572585.682049:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5087 00000100:00100000:3.0:1788572585.701286:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572585.701418:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360728576 00000100:00100000:0.0:1788572585.701434:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360728576 00000100:00100000:0.0:1788572585.701521:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5088 00000100:00100000:0.0:1788572585.701536:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37426:x1875453360728576:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572585.701568:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572585.701605:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bd4 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.701683:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 2/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.701696:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 4/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.701721:0:5837:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.701743:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.701759:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:0.0:1788572585.701763:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572585.701767:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 3/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 248 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572585.701776:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1e00, rc 0) 00000100:00100000:0.0:1788572585.701864:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7849500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+248:37426:x1875453360728576:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 331us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572585.701872:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5088 00010000:00010000:1.0:1788572586.106246:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1788572586.428582:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572586.428595:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572586.428651:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 179 count 11 00000040:00080000:2.0:1788572586.428660:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1788572586.428994:0:11672:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382420736:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1788572586.429391:0:11672:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0750c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572586.434059:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.434462:0:11672:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11672:1875453382420736:192.168.202.40@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1788572586.435388:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360728704 00000100:00100000:0.0:1788572586.435401:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360728704 00000100:00080000:1.0:1788572586.435478:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788572586.435502:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5089 00000100:00100000:0.0:1788572586.435515:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37428:x1875453360728704:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572586.435560:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb4:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572586.435715:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e784a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37428:x1875453360728704:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 201us (1710us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.435723:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5089 00000100:00100000:0.0:1788572586.441724:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.442712:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360728832 00000100:00100000:3.0:1788572586.442729:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360728832 00000100:00100000:3.0:1788572586.442825:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5090 00000100:00100000:3.0:1788572586.442837:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37428:x1875453360728832:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1788572586.442946:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.442971:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.442987:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443055:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443064:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443070:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443110:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443118:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443125:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572586.443463:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00000004:00200000:3.0:1788572586.443469:0:9040:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788572586.443486:0:9040:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443500:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443506:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443513:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443523:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572586.443528:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443536:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443551:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:3.0:1788572586.443554:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572586.443561:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443567:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166a9c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443586:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443591:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443598:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443602:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bc400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572586.443605:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443611:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443624:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x81af725a (ffff9626c4b10700) 00010000:00010000:3.0:1788572586.443627:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572586.443630:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443636:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bc400/0x85ace67feb166aa3 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x81af725a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443653:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443659:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443665:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443669:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bda00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572586.443672:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443678:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443685:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:3.0:1788572586.443687:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788572586.443690:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.443695:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bda00/0x85ace67feb166aaa lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572586.446609:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37428:x1875453360728832:12345-192.168.202.40@tcp:34:lhsmtool_posix.0 Request processed in 3755us (4885us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.446626:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5090 00000100:00100000:1.0:1788572586.467744:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.468820:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360729088 00000100:00100000:0.0:1788572586.468849:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360729088 00000100:00100000:0.0:1788572586.468957:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5091 00000100:00100000:0.0:1788572586.468971:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37428:x1875453360729088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572586.469044:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572586.469073:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166ab1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bdb expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469247:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469269:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469284:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572586.469298:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb4:0x0]: returning size 0 00010000:00010000:0.0:1788572586.469567:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469577:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469586:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469797:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb1800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.469804:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469814:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469829:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:0.0:1788572586.469834:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788572586.469840:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469847:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166ab8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469909:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166ab1 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bdb expref: 69 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469920:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff9627c3da6e00) 00010000:00010000:0.0:1788572586.469934:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1788572586.469939:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166ab1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bdb expref: 69 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.469950:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0200, rc 0) 00000100:00100000:0.0:1788572586.470084:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37428:x1875453360729088:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 1116us (2361us total) trans 4294971175 rc 301/301 00000100:00100000:0.0:1788572586.470103:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5091 00000100:00100000:2.0:1788572586.479661:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.479831:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360729216 00000100:00100000:0.0:1788572586.479854:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360729216 00000100:00100000:0.0:1788572586.479954:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5092 00000100:00100000:0.0:1788572586.479971:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37428:x1875453360729216:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1788572586.480067:0:23058:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xb4:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1788572586.480399:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37428:x1875453360729216:12345-192.168.202.40@tcp:53:lhsmtool_posix.0 Request processed in 430us (1225us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.480408:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5092 00000100:00100000:1.0:1788572586.487692:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.489077:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360729472 00000100:00100000:0.0:1788572586.489108:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360729472 00000100:00100000:0.0:1788572586.489214:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5093 00000100:00100000:0.0:1788572586.489228:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+68:37428:x1875453360729472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572586.489259:0:22536:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572586.489288:0:22536:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166ac6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1be2 expref: -99 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.491626:0:22536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166acd lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.491653:0:22536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166acd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.491668:0:22536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166acd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 22536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788572586.491681:0:22536:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb9:0x0]: returning size 0 00010000:00010000:0.0:1788572586.491754:0:22536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0200/0x85ace67feb166ac6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be2 expref: 71 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.491779:0:22536:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166acd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be2 expref: 70 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.491790:0:22536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae000) 00010000:00010000:0.0:1788572586.491793:0:22536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788572586.491798:0:22536:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb1800, rc 0) 00000100:00100000:0.0:1788572586.491905:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37428:x1875453360729472:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 2678us (4227us total) trans 4294971176 rc 0/0 00000100:00100000:0.0:1788572586.491916:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5093 00000100:00100000:1.0:1788572586.501417:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.502068:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360729728 00000100:00100000:0.0:1788572586.502082:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360729728 00000100:00100000:0.0:1788572586.502183:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5094 00000100:00100000:0.0:1788572586.502196:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37428:x1875453360729728:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1788572586.502281:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572586.502631:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1800/0x85ace67feb166acd lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be2 expref: 70 pid: 22536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.502659:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae000) 00010000:00010000:0.0:1788572586.502672:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788572586.502701:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.502718:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.502729:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.502744:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.502751:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.502758:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503051:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503064:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503071:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503077:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0a00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.503082:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503103:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503113:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:0.0:1788572586.503116:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572586.503120:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503126:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166adb lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503136:0:5839:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503147:0:5839:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503153:0:5839:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503158:0:5839:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0400 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.503162:0:5839:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503169:0:5839:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.503175:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0xe632d3ff (ffff9627dfeae100) 00010000:00010000:0.0:1788572586.503177:0:5839:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572586.504662:0:5839:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.504677:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd0:0x0].0xe632d3ff bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.513131:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37428:x1875453360729728:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 10916us (11707us total) trans 0 rc -2/-2 00000100:00100000:2.0:1788572586.513158:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5094 00000100:00100000:2.0:1788572586.521434:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.521625:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360729856 00000100:00100000:0.0:1788572586.521640:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360729856 00000100:00100000:0.0:1788572586.521717:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5095 00000100:00100000:0.0:1788572586.521730:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+69:37428:x1875453360729856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572586.521762:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572586.521794:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae9 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1be9 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.521842:0:23058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.521860:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.521875:0:23058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 23058 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.522194:0:23058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0400/0x85ace67feb166ae9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 71 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.522218:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 70 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.522227:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:0.0:1788572586.522230:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572586.522234:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0a00, rc 0) 00000100:00100000:0.0:1788572586.522304:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37428:x1875453360729856:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 577us (888us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.522311:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5095 00000100:00100000:1.0:1788572586.529536:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.529888:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360730112 00000100:00100000:2.0:1788572586.529901:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360730112 00000100:00100000:2.0:1788572586.529984:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5096 00000100:00100000:2.0:1788572586.529998:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+70:37428:x1875453360730112:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1788572586.530172:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.530199:0:9040:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.530215:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788572586.530481:0:9040:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e97002f8 reply state 00000000ed2eb7b8 transno 4294971177 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.530494:0:9040:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788572586.531396:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c84ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37428:x1875453360730112:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1391us (1868us total) trans 4294971177 rc 0/0 00000100:00100000:2.0:1788572586.531410:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5096 00010000:00010000:2.0:1788572586.535238:0:3288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.535264:0:3288:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.535272:0:3288:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.535278:0:3288:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8800 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:2.0:1788572586.535283:0:3288:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.535293:0:3288:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.535329:0:3288:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:2.0:1788572586.535333:0:3288:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1788572586.535340:0:3288:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.535347:0:3288:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166af7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572586.542536:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.542734:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360730240 00000100:00100000:2.0:1788572586.542748:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360730240 00000100:00100000:2.0:1788572586.542829:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5097 00000100:00100000:2.0:1788572586.542841:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+71:37428:x1875453360730240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572586.542884:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572586.542912:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bf0 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.542993:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.543040:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 72 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.543051:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:2.0:1788572586.543054:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788572586.543059:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8200, rc 0) 00000100:00100000:2.0:1788572586.543152:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37428:x1875453360730240:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 312us (629us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.543159:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5097 00000100:00100000:1.0:1788572586.550644:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.550782:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360730368 00000100:00100000:3.0:1788572586.550797:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360730368 00000100:00100000:3.0:1788572586.550871:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1115 00000100:00100000:3.0:1788572586.550883:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627ec13cc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37428:x1875453360730368:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1788572586.551542:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627ec13cc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37428:x1875453360730368:12345-192.168.202.40@tcp:49:lhsmtool_posix.0 Request processed in 658us (908us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.551552:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1115 00000100:00100000:1.0:1788572586.560761:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.561510:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360730624 00000100:00100000:2.0:1788572586.561525:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360730624 00000100:00100000:2.0:1788572586.561752:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1116 00000100:00100000:2.0:1788572586.561767:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+72:37428:x1875453360730624:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1788572586.562175:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfdbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37428:x1875453360730624:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 410us (1427us total) trans 4294971178 rc 0/0 00000100:00100000:2.0:1788572586.562185:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1116 00000100:00100000:0.0:1788572586.572363:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.572871:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360730880 00000100:00100000:3.0:1788572586.572886:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360730880 00000100:00100000:3.0:1788572586.572979:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5098 00000100:00100000:3.0:1788572586.572992:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37428:x1875453360730880:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1788572586.573591:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37428:x1875453360730880:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 598us (1239us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.573602:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5098 00000100:00100000:1.0:1788572586.582124:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.582886:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360731136 00000100:00100000:2.0:1788572586.582902:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360731136 00000100:00100000:2.0:1788572586.582996:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5099 00000100:00100000:2.0:1788572586.583051:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37428:x1875453360731136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572586.583080:0:9040:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572586.583117:0:9040:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166b05 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bf7 expref: -99 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.583255:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.583273:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.583283:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.583656:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 74 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.583677:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 74 pid: 5837 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.583694:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2736680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382420864:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572586.583839:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec0750c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572586.585977:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2736680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382420864:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572586.586061:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572586.587167:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.595146:0:5823:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360731264 00000100:00100000:0.0:1788572586.595172:0:5823:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360731264 00010000:00010000:0.0:1788572586.595245:0:5823:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 75 pid: 5837 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.595300:0:5823:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1218 00000100:00100000:0.0:1788572586.595325:0:5823:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:2349:x1875453360731264:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572586.595347:0:5823:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572586.595356:0:5823:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 75 pid: 5837 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.595373:0:5823:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 75 pid: 5837 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.595401:0:5823:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166afe lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf0 expref: 75 pid: 5837 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.595423:0:5823:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:0.0:1788572586.595428:0:5823:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.595440:0:5823:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.595450:0:5823:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.595493:0:5823:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572586.595572:0:5823:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:2349:x1875453360731264:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 261us (8411us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.595580:0:5823:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1218 00010000:00010000:3.0:1788572586.595723:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.595733:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788572586.595761:0:9040:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb9:0x0]: returning size 0 00000004:00080000:3.0:1788572586.596224:0:9040:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x63:0x0], index 0: 0 00010000:00010000:3.0:1788572586.596419:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596429:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596437:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596444:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572586.596450:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596460:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596473:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:3.0:1788572586.596476:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788572586.596481:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596488:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b0c lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596558:0:9040:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166b05 lrc: 1/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf7 expref: 74 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596569:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd0:0x0].0x0 (ffff9627ffc19c00) 00010000:00010000:3.0:1788572586.596572:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788572586.596577:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8800/0x85ace67feb166b05 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd0:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bf7 expref: 74 pid: 9040 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.596587:0:9040:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8800, rc 0) 00000100:00100000:3.0:1788572586.596634:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37428:x1875453360731136:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 13628us (14526us total) trans 4294971179 rc 301/301 00000100:00100000:3.0:1788572586.596647:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5099 00000100:00100000:1.0:1788572586.600310:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.600501:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360731520 00000100:00100000:3.0:1788572586.600515:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360731520 00000100:00100000:3.0:1788572586.600581:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1117 00000100:00100000:3.0:1788572586.600593:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+73:37428:x1875453360731520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1788572586.601123:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb471c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37428:x1875453360731520:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 529us (826us total) trans 4294971180 rc 0/0 00000100:00100000:3.0:1788572586.601133:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1117 00000100:00100000:1.0:1788572586.606580:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.609390:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360731648 00000100:00100000:2.0:1788572586.609410:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360731648 00000100:00100000:2.0:1788572586.609504:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5100 00000100:00100000:2.0:1788572586.609516:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+74:37428:x1875453360731648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1788572586.609542:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788572586.609572:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b1a lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1bfe expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.609669:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166b21 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.609686:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166b21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.609702:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166b21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788572586.609744:0:5837:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb9:0x0]: returning size 0 00000004:00200000:2.0:1788572586.609749:0:5837:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788572586.609771:0:5837:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166b21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.609804:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b1a lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bfe expref: 76 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.609826:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166b21 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bfe expref: 75 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.609836:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:2.0:1788572586.609841:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1788572586.609846:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627e74d8a00, rc 0) 00000100:00100000:2.0:1788572586.610232:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb472680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37428:x1875453360731648:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 716us (3671us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.610241:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5100 00000100:00100000:1.0:1788572586.612655:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572586.613062:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572586.618229:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572586.618267:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.620262:0:5832:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360731776 00000100:00100000:2.0:1788572586.620283:0:5832:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360731776 00000100:00100000:2.0:1788572586.620405:0:5832:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 695 00000100:00100000:2.0:1788572586.620419:0:5832:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0f1ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360731776:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572586.620526:0:5832:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0f1ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360731776:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 109us (2311us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.620536:0:5832:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 695 00000100:00100000:3.0:1788572586.620899:0:13272:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360731904 00000100:00100000:3.0:1788572586.620905:0:13272:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360731904 00000100:00100000:2.0:1788572586.623726:0:6681:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732160 00000100:00100000:2.0:1788572586.623748:0:6681:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732160 00000100:00100000:2.0:1788572586.623796:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 636 00000100:00100000:2.0:1788572586.623808:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360731904:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572586.623909:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9b51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:2348:x1875453360731904:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 103us (11268us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.623918:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 636 00000100:00100000:2.0:1788572586.623958:0:6681:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 637 00000100:00100000:2.0:1788572586.623966:0:6681:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360732160:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572586.623995:0:6681:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360732160:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 30us (10933us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.624001:0:6681:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 637 00000100:00080000:1.0:1788572586.625790:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788572586.625949:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572586.625982:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.626278:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732032 00000100:00100000:3.0:1788572586.626290:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732032 00000100:00100000:3.0:1788572586.626358:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732288 00000100:00100000:3.0:1788572586.626361:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732288 00000100:00100000:3.0:1788572586.626396:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732416 00000100:00100000:3.0:1788572586.626399:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732416 00000100:00100000:3.0:1788572586.626423:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 638 00000100:00100000:3.0:1788572586.626433:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2348:x1875453360732032:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572586.626519:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2348:x1875453360732032:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 87us (8250us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.626528:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 638 00000100:00100000:3.0:1788572586.626551:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 639 00000100:00100000:3.0:1788572586.626557:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360732288:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:3.0:1788572586.626586:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360732288:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 29us (645us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.626597:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 639 00000100:00100000:3.0:1788572586.626611:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1499 00000100:00100000:3.0:1788572586.626618:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2350:x1875453360732416:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1788572586.626632:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788572586.626661:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1c05 expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788572586.626694:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x63:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1788572586.626716:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=934s v=(0 0) 00010000:00010000:3.0:1788572586.626721:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572586.626732:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.626755:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 11 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.626774:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627ff6bda00, rc 0) 00000100:00100000:3.0:1788572586.626796:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f5b64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360732416:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 179us (814us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.626803:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1499 00000100:00080000:3.0:1788572586.626822:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788572586.633219:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.634814:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732672 00000100:00100000:2.0:1788572586.634835:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732672 00000100:00100000:2.0:1788572586.634964:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5101 00000100:00100000:2.0:1788572586.634978:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37428:x1875453360732672:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1788572586.635180:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f596a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37428:x1875453360732672:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 203us (1977us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.635189:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5101 00000100:00100000:1.0:1788572586.668950:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.669034:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732800 00000100:00100000:0.0:1788572586.669054:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732800 00000100:00100000:0.0:1788572586.669153:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1500 00000100:00100000:0.0:1788572586.669166:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:37428:x1875453360732800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572586.669193:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572586.669225:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1c0c expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.669281:0:6682:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.669293:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572586.669299:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572586.669305:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 4/0,0 mode: --/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.669363:0:6682:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.669379:0:6682:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 1034 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.669395:0:6682:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382420992:192.168.202.40@tcp:104: 00000100:00100000:0.0:1788572586.669519:0:6682:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e67cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572586.675517:0:6682:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6682:1875453382420992:192.168.202.40@tcp:104: 00010000:00010000:0.0:1788572586.675570:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 3/0,0 mode: --/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 12 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.675610:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0200, rc 0) 00000100:00100000:0.0:1788572586.675704:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7877b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:37428:x1875453360732800:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 6540us (6769us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.675718:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1500 00000100:00100000:1.0:1788572586.678633:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.678697:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360732928 00000100:00100000:0.0:1788572586.678713:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360732928 00010000:00010000:0.0:1788572586.678766:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 1034 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.678824:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1219 00000100:00100000:0.0:1788572586.678836:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360732928:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572586.678854:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572586.678884:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 1034 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.678902:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 1034 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.679027:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005f7c6fc3/0x85ace67feb166b28 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c05 expref: 12 pid: 6682 timeout: 1034 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.679048:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572586.679053:0:5822:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572586.679065:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.679076:0:5822:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.679211:0:5822:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.679229:0:5822:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382421120:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572586.679552:0:5822:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c3e67cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572586.696158:0:5822:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5822:1875453382421120:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572586.696199:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572586.696276:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7875c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360732928:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 17434us (17650us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.696293:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1219 00000100:00100000:2.0:1788572586.719455:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.719574:0:18208:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360733056 00000100:00100000:3.0:1788572586.719587:0:18208:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360733056 00002000:00100000:3.0:1788572586.719660:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x63:0x0] (0->1048575) req@ffff9626c2736680 x1875453360733056/t0(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:247/0 lens 488/0 e 0 to 0 dl 1788572597 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572586.719679:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2736680 with x1875453360733056 ext(0->1048575) 00002000:00010000:3.0:1788572586.719686:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2736680 00000100:00100000:3.0:1788572586.719778:0:18208:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 394 00000100:00100000:3.0:1788572586.719791:0:18208:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360733056:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1788572586.743818:0:18208:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360733056:12345-192.168.202.40@tcp:4:lhsmtool_posix.0 Request processed in 24013us (24364us total) trans 4294967687 rc 0/0 00000100:00100000:3.0:1788572586.743848:0:18208:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 394 00002000:00100000:3.0:1788572586.743864:0:18208:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x63:0x0] (0->1048575) req@ffff9626c2736680 x1875453360733056/t4294967687(0) o4->9d61336c-09bb-48c3-938e-9c469fbceb0d@192.168.202.40@tcp:247/0 lens 488/448 e 0 to 0 dl 1788572597 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788572586.743876:0:18208:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c2736680 with x1875453360733056 ext(0->1048575) 00002000:00010000:3.0:1788572586.743884:0:18208:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c2736680 00000100:00100000:1.0:1788572586.747495:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.747789:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360733184 00000100:00100000:0.0:1788572586.747803:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360733184 00000100:00100000:0.0:1788572586.747889:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5102 00000100:00100000:0.0:1788572586.747902:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+75:37428:x1875453360733184:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 00000001:00080000:1.0:1788572586.787427:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971176 is committed 00000001:00080000:1.0:1788572586.787470:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971173 is committed 00000020:00080000:1.0:1788572586.787487:0:5859:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 149 batchid updated 00000020:00080000:1.0:1788572586.787501:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 149 00000020:00080000:1.0:1788572586.787521:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 149 00000001:00080000:1.0:1788572586.787530:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971178 is committed 00000001:00080000:1.0:1788572586.787539:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971174 is committed 00000001:00080000:1.0:1788572586.787563:0:5859:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294971180 is committed 00000020:00080000:1.0:1788572586.787582:0:5859:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9627fb17dcc0 refcount 2 committed 0 result 0 batchid 150 00000020:00080000:1.0:1788572586.787588:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c4ffb600 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9627fb17d180 00000020:00080000:1.0:1788572586.787591:0:5859:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9627c4ffb900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9627d57b2800 00000020:00080000:1.0:1788572586.787620:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 150 committed batchid 149 00000020:00080000:1.0:1788572586.789218:0:5870:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 149 00000100:00100000:2.0:1788572586.789787:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37428:x1875453360733184:12345-192.168.202.40@tcp:44:lhsmtool_posix.0 Request processed in 41878us (42298us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.789804:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5102 00000100:00100000:2.0:1788572586.792651:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.792779:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360733312 00000100:00100000:0.0:1788572586.792795:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360733312 00000100:00100000:0.0:1788572586.792882:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1501 00000100:00100000:0.0:1788572586.792894:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+13:2349:x1875453360733312:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1788572586.835714:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967684 is committed 00000001:00080000:1.0:1788572586.835741:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967685 is committed 00000001:00080000:1.0:1788572586.835748:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967686 is committed 00000001:00080000:1.0:1788572586.835756:0:6704:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967687 is committed 00000100:00100000:3.0:1788572586.836211:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360733312:12345-192.168.202.40@tcp:16:lhsmtool_posix.0 Request processed in 43308us (43566us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.836227:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1501 00000100:00100000:1.0:1788572586.839127:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.839710:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360733568 00000100:00100000:3.0:1788572586.839726:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360733568 00000100:00100000:3.0:1788572586.839806:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5103 00000100:00100000:3.0:1788572586.839818:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37428:x1875453360733568:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1788572586.839912:0:5839:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572586.839953:0:5839:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8a00/0x85ace67feb166b21 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bfe expref: 60 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.839978:0:5839:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:3.0:1788572586.839986:0:5839:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788572586.840085:0:5839:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.840104:0:5839:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.840116:0:5839:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788572586.840705:0:5839:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000781f7caf reply state 0000000066e06271 transno 4294971181 ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572586.840718:0:5839:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788572586.840987:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37428:x1875453360733568:12345-192.168.202.40@tcp:36:lhsmtool_posix.0 Request processed in 1168us (1871us total) trans 4294971181 rc 0/0 00000100:00100000:3.0:1788572586.840997:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5103 00000100:00100000:2.0:1788572586.845202:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.845308:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360733696 00000100:00100000:0.0:1788572586.845323:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360733696 00000100:00100000:0.0:1788572586.845398:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5104 00000100:00100000:0.0:1788572586.845412:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37428:x1875453360733696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1788572586.845447:0:23058:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572586.845478:0:23058:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1c13 expref: -99 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.845578:0:23058:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 62 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.845597:0:23058:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 62 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.845626:0:23058:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:0.0:1788572586.845630:0:23058:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572586.845635:0:23058:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0c00, rc 0) 00000100:00100000:0.0:1788572586.845760:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:37428:x1875453360733696:12345-192.168.202.40@tcp:101:lhsmtool_posix.0 Request processed in 352us (936us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.845768:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5104 00010000:00010000:1.0:1788572586.846426:0:3289:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.846440:0:3289:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.846449:0:3289:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.846456:0:3289:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bd000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:1.0:1788572586.846461:0:3289:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.846471:0:3289:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.846489:0:3289:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:1.0:1788572586.846491:0:3289:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788572586.846496:0:3289:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.846502:0:3289:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bd000/0x85ace67feb166b36 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5839 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.849990:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.850232:0:23814:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360733824 00000100:00100000:0.0:1788572586.850247:0:23814:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360733824 00000100:00100000:0.0:1788572586.850429:0:23814:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1502 00000100:00100000:0.0:1788572586.850442:0:23814:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360733824:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1788572586.850797:0:23814:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x63:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1788572586.850801:0:23814:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x63:0x0] updating lvb mtime from disk: 0 -> 1788572578 00002000:00010000:0.0:1788572586.850804:0:23814:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x63:0x0] updating lvb atime from disk: 0 -> 1788572578 00002000:00010000:0.0:1788572586.850807:0:23814:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x63:0x0] updating lvb ctime from disk: 0 -> 1788572578 00002000:00010000:0.0:1788572586.850809:0:23814:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x63:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:0.0:1788572586.850900:0:23814:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2734380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360733824:12345-192.168.202.40@tcp:2:lhsmtool_posix.0 Request processed in 458us (923us total) trans 4294967688 rc 0/0 00000100:00100000:0.0:1788572586.850909:0:23814:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1502 00000100:00100000:3.0:1788572586.853738:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.853903:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734080 00000100:00100000:2.0:1788572586.853916:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734080 00000100:00100000:2.0:1788572586.854052:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5105 00000100:00100000:2.0:1788572586.854066:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:37428:x1875453360734080:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1788572586.854170:0:9040:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.854193:0:9040:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.854205:0:9040:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.854262:0:9040:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.854418:0:9040:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 61 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.854437:0:9040:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2736300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382421248:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572586.854568:0:9040:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572586.857801:0:9040:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2736300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9040:1875453382421248:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572586.857855:0:9040:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572586.861425:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.861742:0:5824:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734208 00000100:00100000:2.0:1788572586.861755:0:5824:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734208 00010000:00010000:2.0:1788572586.861810:0:5824:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.861873:0:5824:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1220 00000100:00100000:2.0:1788572586.861885:0:5824:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2350:x1875453360734208:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572586.861905:0:5824:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788572586.861914:0:5824:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.861931:0:5824:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.861957:0:5824:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0a00/0x85ace67feb166af0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb9:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1be9 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.861978:0:5824:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:2.0:1788572586.861985:0:5824:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.862050:0:5824:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.862060:0:5824:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.862108:0:5824:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788572586.862187:0:5824:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c36ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2350:x1875453360734208:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 304us (776us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.862195:0:5824:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1220 00010000:00010000:3.0:1788572586.862539:0:9040:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862549:0:9040:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862596:0:9040:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862610:0:9040:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862617:0:9040:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862623:0:9040:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8c00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:3.0:1788572586.862627:0:9040:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862635:0:9040:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862646:0:9040:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:3.0:1788572586.862648:0:9040:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1788572586.862652:0:9040:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.862658:0:9040:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8c00/0x85ace67feb166b44 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572586.862708:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627e7848000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37428:x1875453360734080:12345-192.168.202.40@tcp:55:lhsmtool_posix.0 Request processed in 8645us (8984us total) trans 0 rc -22/-22 00000100:00100000:3.0:1788572586.862718:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5105 00000100:00100000:2.0:1788572586.869945:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.870051:0:10416:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734336 00000100:00100000:3.0:1788572586.870064:0:10416:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734336 00000100:00100000:3.0:1788572586.870139:0:10416:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1503 00000100:00100000:3.0:1788572586.870152:0:10416:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2349:x1875453360734336:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1788572586.870217:0:10416:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.870246:0:10416:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.870256:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788572586.870426:0:10416:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572586.870435:0:10416:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 4/1,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.870491:0:10416:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 11 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.870511:0:10416:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 11 pid: 6682 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572586.870531:0:10416:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382421376:192.168.202.40@tcp:104: 00000100:00100000:3.0:1788572586.870696:0:10416:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627fb17d000 going to sleep for 11 seconds 00000100:00100000:0.0:1788572586.875670:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:3.0:1788572586.875812:0:10416:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff962801f91c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:10416:1875453382421376:192.168.202.40@tcp:104: 00010000:00010000:3.0:1788572586.875871:0:10416:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572586.876219:0:23057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734464 00000100:00100000:1.0:1788572586.876230:0:23057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734464 00010000:00010000:3.0:1788572586.876518:0:23057:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 12 pid: 6682 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572586.876579:0:23057:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1221 00000100:00100000:3.0:1788572586.876591:0:23057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+12:2350:x1875453360734464:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788572586.876611:0:23057:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788572586.876644:0:23057:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 12 pid: 6682 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.876661:0:23057:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x63:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 12 pid: 6682 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.876697:0:23057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008692d37d/0x85ace67feb166b2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x63:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c0c expref: 12 pid: 6682 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.876719:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572586.876724:0:23057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1788572586.876731:0:23057:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.876744:0:23057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572586.876825:0:23057:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788572586.877294:0:23057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:9d61336c-09bb-48c3-938e-9c469fbceb0d+11:2350:x1875453360734464:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 700us (1634us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.877304:0:23057:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1221 00010000:00010000:2.0:1788572586.878671:0:10416:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878694:0:10416:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878740:0:10416:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878750:0:10416:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878759:0:10416:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878766:0:10416:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bde00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:2.0:1788572586.878772:0:10416:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878784:0:10416:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878804:0:10416:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.878813:0:10416:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e37eb3a/0x85ace67feb166b4b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 10416 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.879825:0:10416:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2349:x1875453360734336:12345-192.168.202.40@tcp:1:lhsmtool_posix.0 Request processed in 9659us (9878us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.879860:0:10416:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1503 00000100:00100000:0.0:1788572586.883968:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.884235:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734720 00000100:00100000:2.0:1788572586.884248:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734720 00000100:00100000:2.0:1788572586.884430:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 5106 00000100:00100000:2.0:1788572586.884443:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+60:37428:x1875453360734720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1788572586.884617:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.884641:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.884656:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.884957:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.884971:0:5837:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.884980:0:5837:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 4/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.885052:0:5837:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 61 pid: 23058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572586.885069:0:5837:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 61 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572586.885097:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382421504:192.168.202.40@tcp:104: 00000100:00100000:2.0:1788572586.885213:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627ec077000 going to sleep for 11 seconds 00000100:00100000:1.0:1788572586.887930:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.888536:0:5822:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734848 00000100:00100000:2.0:1788572586.888726:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382421504:192.168.202.40@tcp:104: 00010000:00010000:2.0:1788572586.888762:0:5837:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.888803:0:5822:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734848 00010000:00010000:0.0:1788572586.888874:0:5822:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.888918:0:5822:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1222 00000100:00100000:0.0:1788572586.888930:0:5822:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+62:2349:x1875453360734848:12345-192.168.202.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572586.888952:0:5822:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788572586.888960:0:5822:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.888972:0:5822:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.888999:0:5822:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b3d lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c13 expref: 62 pid: 23058 timeout: 1034 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.889042:0:5822:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:0.0:1788572586.889048:0:5822:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.889058:0:5822:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.889066:0:5822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.889158:0:5822:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788572586.889232:0:5822:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627fb1e3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+61:2349:x1875453360734848:12345-192.168.202.40@tcp:103:ldlm_bl.0 Request processed in 303us (1313us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.889240:0:5822:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1222 00010000:00010000:1.0:1788572586.890789:0:5837:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.890808:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788572586.891527:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971182 00000001:00080000:0.0:1788572586.891738:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971183 00000001:00080000:0.0:1788572586.891789:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971184 00000100:00100000:0.0:1788572586.892332:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382421632:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572586.892431:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572586.892477:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627c63ccf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788572586.892696:0:18263:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382421632 00000100:00100000:3.0:1788572586.892708:0:18263:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382421632 00000100:00100000:3.0:1788572586.892863:0:18263:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1788572586.892875:0:18263:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382421632:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1788572586.892985:0:18263:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382421632:12345-0@lo:1000:mdt00_000.0 Request processed in 111us (553us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572586.892994:0:18263:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1788572586.893233:0:5837:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967686 gen 2 00000004:00080000:0.0:1788572586.893240:0:5837:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9627f56f0000, transno 4294967684 00000004:00080000:0.0:1788572586.893257:0:5837:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967686 committed transno 4294967686 00000100:00100000:0.0:1788572586.893295:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382421632:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1788572586.893898:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971185 00000100:00100000:0.0:1788572586.893992:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9627dfff6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382421760:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1788572586.894402:0:5837:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788572586.894861:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32780 going to sleep for 11 seconds 00000100:00080000:1.0:1788572586.895030:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572586.895909:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382421760 00000100:00100000:2.0:1788572586.895920:0:5843:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382421760 00000100:00100000:2.0:1788572586.896130:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:2.0:1788572586.896142:0:5843:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5837:x1875453382421760:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1788572586.896461:0:5843:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c10a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5837:x1875453382421760:12345-0@lo:1000:mdt00_000.0 Request processed in 320us (2058us total) trans 4294967689 rc 0/0 00000100:00100000:2.0:1788572586.896475:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:0.0:1788572586.896579:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9627dfff6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5837:1875453382421760:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1788572586.896712:0:5837:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971186 00010000:00010000:0.0:1788572586.896759:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896772:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896780:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896788:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d9e00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.896794:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896803:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896823:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:0.0:1788572586.896837:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572586.896844:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896851:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d9e00/0x85ace67feb166b59 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896893:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896899:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896905:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896910:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627ff6bce00 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.896914:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896920:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896934:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:0.0:1788572586.896936:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788572586.896948:0:5837:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896962:0:5837:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.896973:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.897040:0:5837:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb1e00/0x85ace67feb166a95 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1bd4 expref: 245 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.897056:0:5837:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382421888:192.168.202.40@tcp:105: 00000100:00100000:0.0:1788572586.897184:0:5837:0:(client.c:2716:ptlrpc_set_wait()) set ffff9627d0f32780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572586.901168:0:5837:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c12ef800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5837:1875453382421888:192.168.202.40@tcp:105: 00010000:00010000:0.0:1788572586.901212:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901225:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627ff6bce00/0x85ace67feb166a8e lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 9040 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901278:0:5837:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901296:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901310:0:5837:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901323:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901330:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901337:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901343:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0000 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.901347:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901354:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901366:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:0.0:1788572586.901369:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788572586.901373:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901379:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0000/0x85ace67feb166b60 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901391:0:5837:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901397:0:5837:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901415:0:5837:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901421:0:5837:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627e74d8200 already unused, calling callback (ffffffffc1ee14f0) 00010000:00010000:0.0:1788572586.901424:0:5837:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901431:0:5837:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901439:0:5837:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb4:0x0].0x0 (ffff9626c4b10f00) 00010000:00010000:0.0:1788572586.901442:0:5837:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1788572586.901445:0:5837:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.901451:0:5837:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627e74d8200/0x85ace67feb166b52 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5837 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572586.902144:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627dfff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37428:x1875453360734720:12345-192.168.202.40@tcp:57:lhsmtool_posix.0 Request processed in 17695us (18185us total) trans 4294971187 rc 0/0 00000100:00100000:0.0:1788572586.902159:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 5106 00000020:00080000:0.0:1788572586.902167:0:5837:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:2.0:1788572586.902889:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1788572586.902900:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1788572586.903937:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 11 00000040:00080000:2.0:1788572586.903954:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1788572586.905894:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.906191:0:14530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360734976 00000100:00100000:2.0:1788572586.906207:0:14530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360734976 00000100:00100000:2.0:1788572586.906294:0:14530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1118 00000100:00100000:2.0:1788572586.906307:0:14530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37426:x1875453360734976:12345-192.168.202.40@tcp:49:md5sum.0 00000100:00100000:2.0:1788572586.906490:0:14530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37426:x1875453360734976:12345-192.168.202.40@tcp:49:md5sum.0 Request processed in 186us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572586.906498:0:14530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1118 00000100:00100000:1.0:1788572586.912227:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.912426:0:5841:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360735232 00000100:00100000:0.0:1788572586.912442:0:5841:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360735232 00000100:00100000:0.0:1788572586.912519:0:5841:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1119 00000100:00100000:0.0:1788572586.912534:0:5841:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+65:37428:x1875453360735232:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1788572586.913446:0:5841:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0xb9:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1788572586.913592:0:5841:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294971188 00010000:00010000:0.0:1788572586.913656:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.913678:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.913693:0:5841:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.913703:0:5841:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.913710:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.913717:0:5841:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.913744:0:5841:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572586.913885:0:5841:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:37428:x1875453360735232:12345-192.168.202.40@tcp:35:lhsmtool_posix.0 Request processed in 1350us (1671us total) trans 4294971189 rc 0/0 00000100:00100000:0.0:1788572586.913896:0:5841:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1119 00000020:00080000:0.0:1788572586.913902:0:5841:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788572586.914181:0:8442:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572586.914190:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.914198:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9627dffb0c00 already unused, calling callback (ffffffffc14d24f0) 00010000:00010000:1.0:1788572586.914203:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.914212:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.914227:0:8442:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb9:0x0].0x0 (ffff9627dfeae100) 00010000:00010000:1.0:1788572586.914230:0:8442:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788572586.914237:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 1/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.914245:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9627dffb0c00/0x85ace67feb166b67 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5841 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572586.914276:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572586.914278:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572586.914280:0:8442:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572586.916814:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.917032:0:6682:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360735360 00000100:00100000:0.0:1788572586.917046:0:6682:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360735360 00000100:00100000:0.0:1788572586.917132:0:6682:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1504 00000100:00100000:0.0:1788572586.917145:0:6682:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+73:37426:x1875453360735360:12345-192.168.202.40@tcp:101:md5sum.0 00010000:00010000:0.0:1788572586.917170:0:6682:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788572586.917198:0:6682:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb166b6e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xc2934ec3fb6f1c1a expref: -99 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788572586.917238:0:6682:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x63:0x0] initial LVB size: 1048576, mtime: 0x6a9b73a2, atime: 0x6a9b73a2, ctime: 0x6a9b73a2, blocks: 0x803 00010000:00010000:0.0:1788572586.917283:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=934s v=(0 0) 00010000:00010000:0.0:1788572586.917289:0:6682:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x63:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788572586.917302:0:6682:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb166b6e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c1a expref: 74 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.917323:0:6682:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ac59cad4/0x85ace67feb166b6e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.202.40@tcp remote: 0xc2934ec3fb6f1c1a expref: 74 pid: 6682 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572586.917338:0:6682:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9627dffb0000, rc 0) 00000100:00100000:0.0:1788572586.917412:0:6682:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:37426:x1875453360735360:12345-192.168.202.40@tcp:101:md5sum.0 Request processed in 269us (613us total) trans 0 rc 0/0 00000100:00100000:0.0:1788572586.917420:0:6682:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1504 00000100:00100000:3.0:1788572586.921621:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:0.0:1788572586.921746:0:18209:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360735488 00000100:00100000:0.0:1788572586.921760:0:18209:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360735488 00002000:00100000:0.0:1788572586.921898:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x63:0x0] (0->32767) req@ffff962801f91c00 x1875453360735488/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:247/0 lens 488/0 e 0 to 0 dl 1788572597 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788572586.921911:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f91c00 with x1875453360735488 ext(0->32767) 00002000:00010000:0.0:1788572586.921919:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff962801f91c00 00000100:00100000:0.0:1788572586.921964:0:18209:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 395 00000100:00100000:0.0:1788572586.921976:0:18209:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360735488:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572586.930818:0:18209:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962801f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2350:x1875453360735488:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 8823us (9265us total) trans 0 rc 32768/32768 00000100:00100000:2.0:1788572586.930859:0:18209:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 395 00002000:00100000:2.0:1788572586.930880:0:18209:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x63:0x0] (0->32767) req@ffff962801f91c00 x1875453360735488/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:247/0 lens 488/440 e 0 to 0 dl 1788572597 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572586.930894:0:18209:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff962801f91c00 with x1875453360735488 ext(0->32767) 00002000:00010000:2.0:1788572586.930904:0:18209:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff962801f91c00 00000100:00100000:0.0:1788572586.938954:0:3297:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:2.0:1788572586.939086:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360735616 00000100:00100000:2.0:1788572586.939100:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360735616 00002000:00100000:2.0:1788572586.939165:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x63:0x0] (32768->1048575) req@ffff9626c12eca80 x1875453360735616/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:247/0 lens 488/0 e 0 to 0 dl 1788572597 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572586.939179:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12eca80 with x1875453360735616 ext(32768->1048575) 00002000:00010000:2.0:1788572586.939186:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c12eca80 00000100:00100000:2.0:1788572586.939224:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 396 00000100:00100000:2.0:1788572586.939234:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2349:x1875453360735616:12345-192.168.202.40@tcp:3:md5sum.0 00000100:00100000:2.0:1788572586.956451:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2349:x1875453360735616:12345-192.168.202.40@tcp:3:md5sum.0 Request processed in 17196us (17489us total) trans 0 rc 1015808/1015808 00000100:00100000:2.0:1788572586.956472:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 396 00002000:00100000:2.0:1788572586.956493:0:6687:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x63:0x0] (32768->1048575) req@ffff9626c12eca80 x1875453360735616/t0(0) o3->a73dcc9b-61e8-4b8a-971c-1141da788d66@192.168.202.40@tcp:247/0 lens 488/440 e 0 to 0 dl 1788572597 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788572586.956506:0:6687:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9626c12eca80 with x1875453360735616 ext(32768->1048575) 00002000:00010000:2.0:1788572586.956516:0:6687:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9626c12eca80 00000100:00100000:3.0:1788572586.965977:0:3296:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572586.966185:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360735872 00000100:00100000:1.0:1788572586.966200:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360735872 00000100:00100000:1.0:1788572586.966278:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 1120 00000100:00100000:1.0:1788572586.966291:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+246:37426:x1875453360735872:12345-192.168.202.40@tcp:35:md5sum.0 00000100:00100000:1.0:1788572586.966651:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627f56f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:37426:x1875453360735872:12345-192.168.202.40@tcp:35:md5sum.0 Request processed in 361us (689us total) trans 4294971190 rc 0/0 00000100:00100000:1.0:1788572586.966659:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 1120 00010000:00010000:2.0:1788572587.132099:0:5825:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572588.162919:0:7615:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572588.348951:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382422016:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788572588.349073:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572588.351051:0:17079:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422016 00000100:00100000:2.0:1788572588.351071:0:17079:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422016 00000100:00100000:2.0:1788572588.351229:0:17079:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1788572588.351244:0:17079:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3304:x1875453382422016:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788572588.351298:0:17079:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 202309632 pending 0 free 3855613952 avail 3853516800 00000100:00100000:2.0:1788572588.351406:0:17079:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3304:x1875453382422016:12345-0@lo:13:osp-pre-0-0.0 Request processed in 167us (2332us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572588.351414:0:17079:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1788572588.351481:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2737800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3304:1875453382422016:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1788572588.354137:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572588.865833:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3305:1875453382422144:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.865951:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572588.866124:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382422272:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866162:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572588.866205:0:23058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422272 00000100:00100000:2.0:1788572588.866215:0:23058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422272 00000100:00100000:2.0:1788572588.866276:0:23058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:2.0:1788572588.866286:0:23058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382422272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866349:0:23058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c48700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1875453382422272:12345-0@lo:400:kworker.0 Request processed in 64us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572588.866358:0:23058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:2.0:1788572588.866413:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3302:1875453382422272:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866449:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3303:1875453382422400:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866476:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572588.866509:0:22536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422400 00000100:00100000:2.0:1788572588.866512:0:22536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422400 00000100:00100000:2.0:1788572588.866549:0:22536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:2.0:1788572588.866556:0:22536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1875453382422400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866590:0:22536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c49c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1875453382422400:12345-0@lo:400:kworker.0 Request processed in 36us (114us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572588.866597:0:22536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00080000:2.0:1788572588.866614:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572588.866632:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3303:1875453382422400:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866660:0:3304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382422528:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866687:0:3304:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572588.866738:0:3305:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382422656:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866767:0:3305:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572588.866820:0:9040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422528 00000100:00100000:2.0:1788572588.866824:0:9040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422528 00000100:00100000:2.0:1788572588.866855:0:9040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:2.0:1788572588.866861:0:9040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382422528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866894:0:9040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1875453382422528:12345-0@lo:400:kworker.0 Request processed in 33us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572588.866900:0:9040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:2.0:1788572588.866920:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422144 00000100:00100000:2.0:1788572588.866924:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422144 00000100:00100000:2.0:1788572588.866948:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:2.0:1788572588.866954:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0f1e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3305:x1875453382422144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.866988:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0f1e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:faeb6f56-8fba-429b-aefd-3a8c55011ac3+19:3305:x1875453382422144:12345-0@lo:400:kworker.0 Request processed in 34us (1039us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572588.866995:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00080000:2.0:1788572588.867061:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788572588.867082:0:3304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3304:1875453382422528:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.867106:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:faeb6f56-8fba-429b-aefd-3a8c55011ac3:3305:1875453382422144:0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.867141:0:20911:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422656 00000100:00100000:2.0:1788572588.867145:0:20911:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422656 00000100:00100000:2.0:1788572588.867177:0:20911:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:2.0:1788572588.867183:0:20911:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c2c4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382422656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788572588.867217:0:20911:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c2c4a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3305:x1875453382422656:12345-0@lo:400:kworker.0 Request processed in 34us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572588.867223:0:20911:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00080000:2.0:1788572588.867235:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788572588.867253:0:3305:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3305:1875453382422656:0@lo:400:kworker.0 00010000:00010000:3.0:1788572589.178436:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572589.370287:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788572589.370314:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9626c2c49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382422784:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788572589.370382:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788572589.370534:0:6683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453382422784 00000100:00100000:2.0:1788572589.370544:0:6683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453382422784 00000100:00100000:2.0:1788572589.370609:0:6683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:2.0:1788572589.370620:0:6683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1875453382422784:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788572589.370661:0:6683:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 202309632 pending 0 free 3853516800 avail 3851419648 00000100:00100000:2.0:1788572589.370736:0:6683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c12eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1875453382422784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572589.370745:0:6683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:0.0:1788572589.370778:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9626c2c49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3303:1875453382422784:0@lo:13:osp-pre-1-0.0 00000040:00080000:0.0:1788572590.010308:0:11672:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1788572590.010319:0:11672:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1788572590.011368:0:11672:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 179 count 6 00000040:00080000:0.0:1788572590.011379:0:11672:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1788572590.203501:0:8441:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1788572590.462781:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967686 is committed 00000001:00080000:1.0:1788572590.463125:0:7629:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967685 is committed 00010000:00010000:2.0:1788572591.230188:0:7616:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572591.732910:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.732951:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.732989:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.733059:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.733083:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.733103:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.733149:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.40@tcp (source: 12345-192.168.202.40@tcp) 00000100:00100000:1.0:1788572591.733198:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736256 00000100:00100000:1.0:1788572591.733208:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736256 00000100:00100000:1.0:1788572591.733269:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736768 00000100:00100000:1.0:1788572591.733272:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736768 00000100:00100000:1.0:1788572591.733295:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736384 00000100:00100000:1.0:1788572591.733298:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736384 00000100:00100000:1.0:1788572591.733317:0:6679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736640 00000100:00100000:1.0:1788572591.733320:0:6679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736640 00000100:00100000:1.0:1788572591.733341:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 641 00000100:00100000:1.0:1788572591.733350:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2348:x1875453360736256:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572591.733441:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+74:2348:x1875453360736256:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 92us (489us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572591.733450:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 641 00000100:00100000:1.0:1788572591.733469:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 642 00000100:00100000:1.0:1788572591.733475:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360736768:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572591.733512:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+9:2348:x1875453360736768:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 26us (442us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572591.733520:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 642 00000100:00100000:1.0:1788572591.733532:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 643 00000100:00100000:1.0:1788572591.733540:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2348:x1875453360736384:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572591.733564:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a73dcc9b-61e8-4b8a-971c-1141da788d66+85:2348:x1875453360736384:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 26us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572591.733571:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 643 00000100:00100000:1.0:1788572591.733582:0:6679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 644 00000100:00100000:1.0:1788572591.733589:0:6679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360736640:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572591.733611:0:6679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9626c1d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:9d61336c-09bb-48c3-938e-9c469fbceb0d+10:2348:x1875453360736640:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 23us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572591.733618:0:6679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 644 00000100:00100000:1.0:1788572591.733639:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736512 00000100:00100000:1.0:1788572591.733643:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736512 00000100:00100000:1.0:1788572591.733671:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736128 00000100:00100000:1.0:1788572591.733674:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736128 00000100:00100000:1.0:1788572591.733693:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 667 00000100:00100000:1.0:1788572591.733699:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2348:x1875453360736512:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572591.733731:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9d61336c-09bb-48c3-938e-9c469fbceb0d+67:2348:x1875453360736512:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 33us (742us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572591.733739:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 667 00000100:00100000:1.0:1788572591.733752:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 668 00000100:00100000:1.0:1788572591.733759:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2348:x1875453360736128:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:1.0:1788572591.733787:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627c9a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a73dcc9b-61e8-4b8a-971c-1141da788d66+247:2348:x1875453360736128:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 29us (704us total) trans 0 rc 0/0 00000100:00100000:1.0:1788572591.733793:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 668 00000100:00080000:1.0:1788572591.733810:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1788572591.733815:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788572591.733817:0:5833:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788572591.735179:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875453360736000 00000100:00100000:2.0:1788572591.735189:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875453360736000 00000100:00100000:2.0:1788572591.735238:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.40@tcp, seq: 697 00000100:00100000:2.0:1788572591.735247:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9627d0f1e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360736000:12345-192.168.202.40@tcp:400:kworker.0 00000100:00100000:2.0:1788572591.735295:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9627d0f1e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:e5ae0868-1fdf-4cd5-a61f-2f9ccc0c273d+13:2348:x1875453360736000:12345-192.168.202.40@tcp:400:kworker.0 Request processed in 50us (2409us total) trans 0 rc 0/0 00000100:00100000:2.0:1788572591.735303:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.40@tcp, seq: 697 Debug log: 64955 lines, 64955 kept, 0 dropped, 0 bad.